[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:38 dist-test-slave-x4qh krb5kdc[27731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:38 dist-test-slave-x4qh krb5kdc[27731](info): set up 2 sockets
Aug 08 18:17:38 dist-test-slave-x4qh krb5kdc[27731](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 18:17:39.682652 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:39 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:39.800298 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42973
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:39.915447 27756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:39.915719 27756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:39.915796 27756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:39.919763 27756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:39.919832 27756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:39.919867 27756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:39.919919 27756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:39.919955 27756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:39.925333 27756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:39.926831 27756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:39.928189 27756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:39.940721 27771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:39.941007 27766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:39.941784 27756 server_base.cc:1034] running on GCE node
W20260808 18:17:39.943517 27768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:39.944386 27756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:39.946444 27756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:39.949702 27756 hybrid_clock.cc:648] HybridClock initialized: now 1786213059949616 us; error 133 us; skew 500 ppm
Aug 08 18:17:39 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213059, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:39.952960 27756 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:39.954053 27756 webserver.cc:460] Webserver started at http://127.27.18.254:43627/ using document root <none> and password file <none>
I20260808 18:17:39.954895 27756 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:39.954957 27756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:39.955319 27756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:39.957859 27756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "c5be60d9186e4c008ac3e4317d51c32f"
format_stamp: "Formatted at 2026-08-08 18:17:39 on dist-test-slave-x4qh"
server_key: "f9b8a10b4bf1439488854cfd412b22d0"
server_key_iv: "d0c532e6a055e2e8e1258174bf0efc53"
server_key_version: "encryptionkey@0"
I20260808 18:17:39.958609 27756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "c5be60d9186e4c008ac3e4317d51c32f"
format_stamp: "Formatted at 2026-08-08 18:17:39 on dist-test-slave-x4qh"
server_key: "f9b8a10b4bf1439488854cfd412b22d0"
server_key_iv: "d0c532e6a055e2e8e1258174bf0efc53"
server_key_version: "encryptionkey@0"
I20260808 18:17:39.963231 27756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 18:17:39.966606 27786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:39.968250 27756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:17:39.968448 27756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "c5be60d9186e4c008ac3e4317d51c32f"
format_stamp: "Formatted at 2026-08-08 18:17:39 on dist-test-slave-x4qh"
server_key: "f9b8a10b4bf1439488854cfd412b22d0"
server_key_iv: "d0c532e6a055e2e8e1258174bf0efc53"
server_key_version: "encryptionkey@0"
I20260808 18:17:39.968546 27756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:39.980225 27756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:39.983125 27756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:39.983354 27756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:39.991921 27756 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:42973
I20260808 18:17:39.991971 27867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:42973 every 8 connection(s)
I20260808 18:17:39.993136 27756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:39.996392 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27756
I20260808 18:17:39.996510 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:39.997853 27723 external_mini_cluster.cc:1428] Setting key d3928b2161db69bea2af66d76b0108fa
I20260808 18:17:39.997915 27869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:40.006109 27869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f: Bootstrap starting.
I20260808 18:17:40.009650 27869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:40.011065 27869 log.cc:824] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:40.013799 27869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f: No bootstrap required, opened a new log
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:40.019501 27869 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5be60d9186e4c008ac3e4317d51c32f" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42973 } }
I20260808 18:17:40.019904 27869 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:40.020012 27869 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5be60d9186e4c008ac3e4317d51c32f, State: Initialized, Role: FOLLOWER
I20260808 18:17:40.020901 27869 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5be60d9186e4c008ac3e4317d51c32f" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42973 } }
I20260808 18:17:40.021144 27869 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:40.021268 27869 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:40.021401 27869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:40.022533 27869 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5be60d9186e4c008ac3e4317d51c32f" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42973 } }
I20260808 18:17:40.023144 27869 leader_election.cc:302] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5be60d9186e4c008ac3e4317d51c32f; no voters:
I20260808 18:17:40.023600 27869 leader_election.cc:288] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:40.023727 27876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:40.023942 27876 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [term 1 LEADER]: Becoming Leader. State: Replica: c5be60d9186e4c008ac3e4317d51c32f, State: Running, Role: LEADER
I20260808 18:17:40.024333 27876 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5be60d9186e4c008ac3e4317d51c32f" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42973 } }
I20260808 18:17:40.025338 27869 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:40.027036 27878 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5be60d9186e4c008ac3e4317d51c32f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5be60d9186e4c008ac3e4317d51c32f" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42973 } } }
I20260808 18:17:40.027187 27878 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:40.027036 27880 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5be60d9186e4c008ac3e4317d51c32f. Latest consensus state: current_term: 1 leader_uuid: "c5be60d9186e4c008ac3e4317d51c32f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5be60d9186e4c008ac3e4317d51c32f" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42973 } } }
I20260808 18:17:40.027307 27880 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:40.027352 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.004226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58988 (local address 127.27.18.254:42973)
0808 18:17:40.004718 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:40.004735 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:40.004822 (+ 87us) server_negotiation.cc:407] Connection header received
0808 18:17:40.006795 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:40.006818 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:40.007168 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:40.007509 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:40.009846 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.011726 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.013010 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.013800 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.013980 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.018684 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:40.018735 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:40.018764 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:40.018799 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:40.021869 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.022428 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.022436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.022442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.022582 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.022966 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.022969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.022971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.023638 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:40.023828 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:40.025908 (+ 2080us) server_negotiation.cc:299] Negotiation successful
0808 18:17:40.026196 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":162,"threads_started":1}
W20260808 18:17:40.033002 27901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:17:40.033174 27901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:17:40.033275 27896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:40.034238 27896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:40.041071 27896 catalog_manager.cc:1347] Generated new cluster ID: 0444b7b3d58b497caa3ce5d70bd6c32a
I20260808 18:17:40.041136 27896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:40.053279 27896 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:40.054157 27896 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:40.060016 27896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5be60d9186e4c008ac3e4317d51c32f: Generated new TSK 0
I20260808 18:17:40.060830 27896 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:17:40.097785 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42973
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:40.219481 27911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:40.219745 27911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:40.219826 27911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:40.223800 27911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:40.223871 27911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:40.223953 27911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:17:40.229395 27911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:40.230365 27911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:40.231321 27911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:40.241161 27924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:40.241178 27918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:40.241348 27911 server_base.cc:1034] running on GCE node
W20260808 18:17:40.241573 27919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:40.241977 27911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:40.242586 27911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:40.243763 27911 hybrid_clock.cc:648] HybridClock initialized: now 1786213060243733 us; error 45 us; skew 500 ppm
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:40.246476 27911 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:17:40.247328 27911 webserver.cc:460] Webserver started at http://127.27.18.193:33165/ using document root <none> and password file <none>
I20260808 18:17:40.247898 27911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:40.247961 27911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:40.248121 27911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:40.249974 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "a6c21b4fd4c14f65a975ca6dd7718035"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "fbcd68cfaf7b58f6b0bdb7df104a90d2"
server_key_iv: "b775930c162392cb77cbbcc4183c3003"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.250478 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "a6c21b4fd4c14f65a975ca6dd7718035"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "fbcd68cfaf7b58f6b0bdb7df104a90d2"
server_key_iv: "b775930c162392cb77cbbcc4183c3003"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.254019 27911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:17:40.256197 27938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.257301 27911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:17:40.257448 27911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "a6c21b4fd4c14f65a975ca6dd7718035"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "fbcd68cfaf7b58f6b0bdb7df104a90d2"
server_key_iv: "b775930c162392cb77cbbcc4183c3003"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.257557 27911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:40.281831 27911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:40.284713 27911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:40.284907 27911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:40.285560 27911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:40.286665 27911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:40.286754 27911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.286826 27911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:40.286872 27911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.296593 27911 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:41861
I20260808 18:17:40.296638 28120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:41861 every 8 connection(s)
I20260808 18:17:40.297433 27911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:17:40.304126 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27911
I20260808 18:17:40.304224 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:17:40.304491 27723 external_mini_cluster.cc:1428] Setting key d1e742e5855172dc9a979df53a60baf8
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:40.311169 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.299546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:49697 (local address 127.27.18.254:42973)
0808 18:17:40.299683 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:40.299688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:40.300438 (+ 750us) server_negotiation.cc:407] Connection header received
0808 18:17:40.301900 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:40.301903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:40.301959 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:40.302068 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:40.303185 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.303843 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.305243 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.305483 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.305564 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.307991 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:40.307994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:40.307997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:40.308027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:40.309254 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.309731 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.309735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.309737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.309820 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.310134 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.310139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.310140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.310312 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:40.310414 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:40.310887 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 18:17:40.311007 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:17:40.311951 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.299861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42973 (local address 127.27.18.193:49697)
0808 18:17:40.300291 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:40.300331 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:40.301693 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:40.302246 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:40.302252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:40.302591 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:40.303025 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:40.303039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.303987 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.303991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:40.305123 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:40.305133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.305635 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.305639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:40.305874 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.306453 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:40.306468 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:40.307839 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:40.309374 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:40.309379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:40.309391 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:40.309630 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:40.309908 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:40.309912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:40.309913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:40.310043 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:40.310414 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:40.310421 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:40.310660 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 18:17:40.310841 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 18:17:40.311133 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":105,"threads_started":1}
I20260808 18:17:40.313283 28121 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42973
I20260808 18:17:40.313539 28121 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:40.313994 28121 heartbeater.cc:511] Master 127.27.18.254:42973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:17:40.315994 27816 ts_manager.cc:194] Registered new tserver with Master: a6c21b4fd4c14f65a975ca6dd7718035 (127.27.18.193:41861)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:17:40.317616 27816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:49697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:17:40.331960 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42973
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:40.438210 28129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:40.438458 28129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:40.438539 28129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:40.442484 28129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:40.442564 28129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:40.442646 28129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:17:40.447645 28129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:40.448527 28129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:40.449409 28129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:40.458357 28145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:40.458406 28138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:40.458380 28129 server_base.cc:1034] running on GCE node
W20260808 18:17:40.458534 28140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:40.458897 28129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:40.459498 28129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:40.460651 28129 hybrid_clock.cc:648] HybridClock initialized: now 1786213060460635 us; error 28 us; skew 500 ppm
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:40.463313 28129 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:17:40.464166 28129 webserver.cc:460] Webserver started at http://127.27.18.194:37071/ using document root <none> and password file <none>
I20260808 18:17:40.464645 28129 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:40.464701 28129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:40.464854 28129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:40.466598 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "8688e63376de44018918508e3d4298f6"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "6b3f8ff73927cbdf89f184e18d6d0c5d"
server_key_iv: "c354b278293021d3d42232562bd22df9"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.467070 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "8688e63376de44018918508e3d4298f6"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "6b3f8ff73927cbdf89f184e18d6d0c5d"
server_key_iv: "c354b278293021d3d42232562bd22df9"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.470592 28129 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:17:40.473240 28155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.474385 28129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:40.474476 28129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "8688e63376de44018918508e3d4298f6"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "6b3f8ff73927cbdf89f184e18d6d0c5d"
server_key_iv: "c354b278293021d3d42232562bd22df9"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.474555 28129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:40.494827 28129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:40.497370 28129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:40.497506 28129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:40.497987 28129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:40.498697 28129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:40.498745 28129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.498780 28129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:40.498795 28129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.508153 28129 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:34491
I20260808 18:17:40.508242 28324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:34491 every 8 connection(s)
I20260808 18:17:40.509161 28129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:17:40.517359 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28129
I20260808 18:17:40.517480 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:17:40.517776 27723 external_mini_cluster.cc:1428] Setting key 4115a5dd130de1f5a3dbaecba7472677
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:40.528205 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.511253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:55723 (local address 127.27.18.254:42973)
0808 18:17:40.511412 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:40.511417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:40.512336 (+ 919us) server_negotiation.cc:407] Connection header received
0808 18:17:40.514126 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:40.514132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:40.514203 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:40.514269 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:40.516308 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.516990 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.518858 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.519195 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.519256 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.522008 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:40.522014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:40.522018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:40.522046 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:40.523848 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.526043 (+ 2195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.526047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.526049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.526144 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.526643 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.526649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.526654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.526951 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:40.527091 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:40.527781 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 18:17:40.527942 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:17:40.528764 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.511682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42973 (local address 127.27.18.194:55723)
0808 18:17:40.512161 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:40.512219 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:40.513857 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:40.514767 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:40.514781 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:40.515428 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:40.516168 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:40.516183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.517130 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.517133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:40.518682 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:40.518699 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.519500 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.519503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:40.519638 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.520167 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:40.520182 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:40.521825 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:40.523980 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:40.523987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:40.524002 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:40.524358 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:40.526288 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:40.526294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:40.526297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:40.526488 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:40.527107 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:40.527117 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:40.527432 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 18:17:40.527698 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 18:17:40.527954 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":106,"threads_started":1}
I20260808 18:17:40.530050 28326 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42973
I20260808 18:17:40.530336 28326 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:40.530922 28326 heartbeater.cc:511] Master 127.27.18.254:42973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
I20260808 18:17:40.532105 27816 ts_manager.cc:194] Registered new tserver with Master: 8688e63376de44018918508e3d4298f6 (127.27.18.194:34491)
I20260808 18:17:40.532670 27816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:55723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:17:40.551724 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42973
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:40.657195 28335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:40.657416 28335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:40.657459 28335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:40.661252 28335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:40.661295 28335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:40.661396 28335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:17:40.666229 28335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:40.667138 28335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:40.668069 28335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:40.677312 28347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:40.677395 28341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:40.677395 28340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:40.677440 28335 server_base.cc:1034] running on GCE node
I20260808 18:17:40.677934 28335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:40.678463 28335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:40.679611 28335 hybrid_clock.cc:648] HybridClock initialized: now 1786213060679589 us; error 32 us; skew 500 ppm
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:40.681983 28335 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:17:40.682818 28335 webserver.cc:460] Webserver started at http://127.27.18.195:40225/ using document root <none> and password file <none>
I20260808 18:17:40.683322 28335 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:40.683449 28335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:40.683662 28335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:40.685456 28335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "8144bdd755324777b4edab9943cd5777"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "57c42976d8cd371c5cbf551e8f8af631"
server_key_iv: "b8e768148a28975b9ffd5fcb14cb6a4a"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.685981 28335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "8144bdd755324777b4edab9943cd5777"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "57c42976d8cd371c5cbf551e8f8af631"
server_key_iv: "b8e768148a28975b9ffd5fcb14cb6a4a"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.689440 28335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:17:40.691599 28359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.692664 28335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:17:40.692813 28335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "8144bdd755324777b4edab9943cd5777"
format_stamp: "Formatted at 2026-08-08 18:17:40 on dist-test-slave-x4qh"
server_key: "57c42976d8cd371c5cbf551e8f8af631"
server_key_iv: "b8e768148a28975b9ffd5fcb14cb6a4a"
server_key_version: "encryptionkey@0"
I20260808 18:17:40.692924 28335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:40.708768 28335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:40.711017 28335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:40.711217 28335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:40.711802 28335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:40.712584 28335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:40.712654 28335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.712720 28335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:40.712767 28335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:40.721256 28335 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:33063
I20260808 18:17:40.721278 28552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:33063 every 8 connection(s)
I20260808 18:17:40.721966 28335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:17:40.727205 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28335
I20260808 18:17:40.727317 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:17:40.727602 27723 external_mini_cluster.cc:1428] Setting key 7dee035cf2e71d3676957f34a5a0dc1b
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:40.734964 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.723630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:57555 (local address 127.27.18.254:42973)
0808 18:17:40.723754 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:40.723758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:40.724637 (+ 879us) server_negotiation.cc:407] Connection header received
0808 18:17:40.725917 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:40.725920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:40.725981 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:40.726076 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:40.727156 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.727907 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.729283 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.729523 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.729628 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.731851 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:40.731855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:40.731858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:40.731883 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:40.733063 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.733544 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.733547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.733549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.733627 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.733961 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.733965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.733966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.734154 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:40.734260 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:40.734688 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 18:17:40.734807 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:17:40.735661 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.723907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42973 (local address 127.27.18.195:57555)
0808 18:17:40.724439 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:40.724475 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:40.725721 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:40.726235 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:40.726241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:40.726574 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:40.726980 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:40.727016 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.728024 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.728028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:40.729154 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:40.729164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.729675 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.729679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:40.729839 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.730378 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:40.730392 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:40.731693 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:40.733176 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:40.733182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:40.733194 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:40.733435 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:40.733717 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:40.733721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:40.733723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:40.733863 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:40.734260 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:40.734266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:40.734474 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 18:17:40.734645 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 18:17:40.734931 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":204,"threads_started":1}
I20260808 18:17:40.736641 28553 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42973
I20260808 18:17:40.736829 28553 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:40.737200 28553 heartbeater.cc:511] Master 127.27.18.254:42973 requested a full tablet report, sending...
I20260808 18:17:40.738095 27816 ts_manager.cc:194] Registered new tserver with Master: 8144bdd755324777b4edab9943cd5777 (127.27.18.195:33063)
I20260808 18:17:40.738621 27816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:57555
I20260808 18:17:40.741770 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:17:40.767930 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.758896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59596 (local address 127.27.18.193:41861)
0808 18:17:40.759156 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:40.759173 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:40.759197 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:17:40.759343 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:40.759355 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:40.759566 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:40.759705 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:40.760449 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.761318 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.762213 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.762568 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.762685 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.764963 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:40.764971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:40.764981 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:40.765004 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:40.766351 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.766733 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.766737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.766738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.766814 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.767047 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.767049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.767050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.767258 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:40.767472 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:40.767666 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 18:17:40.767785 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":60,"threads_started":1}
W20260808 18:17:40.768859 27988 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59596
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[27731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:17:40.792115 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:40.785211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.27.18.193:41861)
0808 18:17:40.785419 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:40.785422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:40.785437 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:17:40.785555 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:40.785557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:40.785589 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:40.785652 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:40.786223 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.786700 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.787420 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:40.787617 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:40.787687 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:40.789617 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:40.789619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:40.789621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:40.789640 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:40.790900 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.791215 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.791217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.791218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.791278 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:40.791526 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:40.791528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:40.791529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:40.791655 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:40.791741 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:40.791945 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 18:17:40.792024 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 18:17:40.793115 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27911
I20260808 18:17:40.808830 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28129
I20260808 18:17:40.817571 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28335
I20260808 18:17:40.828051 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27756
2026-08-08T18:17:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2394 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[28574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[28574](info): set up 2 sockets
Aug 08 18:17:40 dist-test-slave-x4qh krb5kdc[28574](info): commencing operation
krb5kdc: starting...
W20260808 18:17:42.022193 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:42.134021 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:43011
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:43011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:42.238080 28598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:42.238335 28598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:42.238402 28598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:42.242076 28598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:42.242154 28598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:42.242192 28598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:42.242249 28598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:42.242285 28598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:42.247241 28598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:43011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:43011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:42.248248 28598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:42.249213 28598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:42.257568 28607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.257573 28614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.257584 28608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:42.257977 28598 server_base.cc:1034] running on GCE node
I20260808 18:17:42.258526 28598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:42.259581 28598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:42.260739 28598 hybrid_clock.cc:648] HybridClock initialized: now 1786213062260721 us; error 29 us; skew 500 ppm
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:42.266392 28598 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:42.267249 28598 webserver.cc:460] Webserver started at http://127.27.18.254:32815/ using document root <none> and password file <none>
I20260808 18:17:42.267827 28598 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:42.267915 28598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:42.268137 28598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:42.270073 28598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "cfa01917b86745ef87ce94dc3c97a851"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "3dc48b38794e20d6329a0444539ec060"
server_key_iv: "4784c820704949573a593e95f7666104"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.270634 28598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "cfa01917b86745ef87ce94dc3c97a851"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "3dc48b38794e20d6329a0444539ec060"
server_key_iv: "4784c820704949573a593e95f7666104"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.274152 28598 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:17:42.276285 28624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.277396 28598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:42.277534 28598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "cfa01917b86745ef87ce94dc3c97a851"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "3dc48b38794e20d6329a0444539ec060"
server_key_iv: "4784c820704949573a593e95f7666104"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.277642 28598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:42.315553 28598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:42.318245 28598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:42.318450 28598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:42.325910 28598 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:43011
I20260808 18:17:42.325945 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:43011 every 8 connection(s)
I20260808 18:17:42.326823 28598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:42.329278 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28598
I20260808 18:17:42.329396 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:42.329675 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:42.329691 27723 external_mini_cluster.cc:1428] Setting key 17eea11253640afc18b02e6e79b4ea4a
I20260808 18:17:42.336212 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851: Bootstrap starting.
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:42.338771 28714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:42.339749 28714 log.cc:824] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:42.342046 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851: No bootstrap required, opened a new log
I20260808 18:17:42.343899 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:42.331096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.27.18.254:43011)
0808 18:17:42.331569 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:42.331583 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:42.331616 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:17:42.332444 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:42.332462 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:42.332764 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:42.333075 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:42.333901 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.334912 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.335820 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.336312 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.336498 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:42.339227 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:42.339232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:42.339243 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:42.339267 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:42.341091 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:42.341596 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:42.341601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:42.341605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:42.341707 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:42.342022 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:42.342024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:42.342025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:42.342428 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:42.342569 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:42.342891 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 18:17:42.343176 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":147,"threads_started":1}
I20260808 18:17:42.346489 28714 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa01917b86745ef87ce94dc3c97a851" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43011 } }
I20260808 18:17:42.346710 28714 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:42.346796 28714 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa01917b86745ef87ce94dc3c97a851, State: Initialized, Role: FOLLOWER
I20260808 18:17:42.347236 28714 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa01917b86745ef87ce94dc3c97a851" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43011 } }
I20260808 18:17:42.347409 28714 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:42.347503 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:42.347643 28714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:42.348634 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa01917b86745ef87ce94dc3c97a851" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43011 } }
I20260808 18:17:42.349030 28714 leader_election.cc:302] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa01917b86745ef87ce94dc3c97a851; no voters:
I20260808 18:17:42.349392 28714 leader_election.cc:288] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:42.349481 28722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:42.349751 28722 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [term 1 LEADER]: Becoming Leader. State: Replica: cfa01917b86745ef87ce94dc3c97a851, State: Running, Role: LEADER
I20260808 18:17:42.350108 28722 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa01917b86745ef87ce94dc3c97a851" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43011 } }
I20260808 18:17:42.350636 28714 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:42.351923 28724 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa01917b86745ef87ce94dc3c97a851. Latest consensus state: current_term: 1 leader_uuid: "cfa01917b86745ef87ce94dc3c97a851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa01917b86745ef87ce94dc3c97a851" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43011 } } }
I20260808 18:17:42.351933 28723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfa01917b86745ef87ce94dc3c97a851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa01917b86745ef87ce94dc3c97a851" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43011 } } }
I20260808 18:17:42.352102 28724 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:42.352101 28723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:42.352949 28736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:42.355633 28736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:42.360451 28736 catalog_manager.cc:1347] Generated new cluster ID: 2eb538af8566454690092e7140515f34
I20260808 18:17:42.360515 28736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:42.367947 28736 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:42.368755 28736 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:42.374508 28736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfa01917b86745ef87ce94dc3c97a851: Generated new TSK 0
I20260808 18:17:42.375093 28736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:17:42.403925 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:43011
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:42.509632 28754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:42.509857 28754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:42.509894 28754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:42.513576 28754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:42.513624 28754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:42.513695 28754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:17:42.518337 28754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:43011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:42.519179 28754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:42.520016 28754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:42.528875 28769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.528913 28772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.528885 28767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:42.529217 28754 server_base.cc:1034] running on GCE node
I20260808 18:17:42.529590 28754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:42.530189 28754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:42.531344 28754 hybrid_clock.cc:648] HybridClock initialized: now 1786213062531321 us; error 35 us; skew 500 ppm
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:42.533970 28754 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:17:42.534719 28754 webserver.cc:460] Webserver started at http://127.27.18.193:38211/ using document root <none> and password file <none>
I20260808 18:17:42.535221 28754 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:42.535279 28754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:42.535544 28754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:42.537313 28754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "20bad0ca1b3446b49d80f5a171eddc6e"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "74f7f18938d40718d31adf2cba6f3cc7"
server_key_iv: "b8792f11f55f8b36f4ba7ee8bbda19c2"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.537838 28754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "20bad0ca1b3446b49d80f5a171eddc6e"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "74f7f18938d40718d31adf2cba6f3cc7"
server_key_iv: "b8792f11f55f8b36f4ba7ee8bbda19c2"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.541379 28754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:17:42.543586 28781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.544636 28754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:42.544764 28754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "20bad0ca1b3446b49d80f5a171eddc6e"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "74f7f18938d40718d31adf2cba6f3cc7"
server_key_iv: "b8792f11f55f8b36f4ba7ee8bbda19c2"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.544868 28754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:42.568555 28754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:42.571594 28754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:42.571795 28754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:42.572373 28754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:42.573149 28754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:42.573220 28754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.573292 28754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:42.573361 28754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.581992 28754 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:42605
I20260808 18:17:42.582068 28983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:42605 every 8 connection(s)
I20260808 18:17:42.582839 28754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:17:42.589619 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28754
I20260808 18:17:42.589725 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:17:42.590026 27723 external_mini_cluster.cc:1428] Setting key 5edddba312fe2d32f930f506904516ed
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:42.595999 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:42.584514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:32831 (local address 127.27.18.254:43011)
0808 18:17:42.584653 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:42.584658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:42.585441 (+ 783us) server_negotiation.cc:407] Connection header received
0808 18:17:42.586680 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:42.586683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:42.586730 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:42.586810 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:42.587899 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.588422 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.589515 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.589896 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.590014 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:42.592418 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:42.592425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:42.592431 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:42.592496 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:42.593988 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:42.594487 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:42.594491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:42.594493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:42.594571 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:42.594910 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:42.594915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:42.594916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:42.595083 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:42.595186 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:42.595721 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 18:17:42.595838 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:17:42.596772 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:42.584887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43011 (local address 127.27.18.193:32831)
0808 18:17:42.585293 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:42.585341 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:42.586496 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:42.586969 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:42.586975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:42.587276 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:42.587779 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:42.587791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.588531 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.588533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:42.589391 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:42.589399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.590044 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.590047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:42.590230 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:42.590782 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:42.590798 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:42.592243 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:42.594117 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:42.594123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:42.594135 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:42.594383 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:42.594664 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:42.594668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:42.594670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:42.594812 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:42.595200 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:42.595208 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:42.595499 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 18:17:42.595678 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 18:17:42.595966 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":108,"threads_started":1}
I20260808 18:17:42.598110 28984 heartbeater.cc:347] Connected to a master server at 127.27.18.254:43011
I20260808 18:17:42.598335 28984 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:42.598767 28984 heartbeater.cc:511] Master 127.27.18.254:43011 requested a full tablet report, sending...
I20260808 18:17:42.600510 28650 ts_manager.cc:194] Registered new tserver with Master: 20bad0ca1b3446b49d80f5a171eddc6e (127.27.18.193:42605)
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:17:42.601773 28650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:32831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:17:42.618513 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:43011
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:42.722313 28990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:42.722543 28990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:42.722584 28990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:42.726523 28990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:42.726567 28990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:42.726616 28990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:17:42.731498 28990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:43011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:42.732357 28990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:42.733247 28990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:42.742269 29001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.742383 29002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.742269 29006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:42.742508 28990 server_base.cc:1034] running on GCE node
I20260808 18:17:42.743093 28990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:42.743723 28990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:42.744880 28990 hybrid_clock.cc:648] HybridClock initialized: now 1786213062744853 us; error 38 us; skew 500 ppm
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:42.747601 28990 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:17:42.748503 28990 webserver.cc:460] Webserver started at http://127.27.18.194:37519/ using document root <none> and password file <none>
I20260808 18:17:42.749024 28990 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:42.749115 28990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:42.749338 28990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:42.751464 28990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "b367f7182fcb4b7f94633abd82a3a2fe"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "15f21e3e4014dfef5a9fad1434795dd0"
server_key_iv: "f7298d3dadf3a13440de2ef88d03cb22"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.752019 28990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "b367f7182fcb4b7f94633abd82a3a2fe"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "15f21e3e4014dfef5a9fad1434795dd0"
server_key_iv: "f7298d3dadf3a13440de2ef88d03cb22"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.755496 28990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:17:42.757692 29017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.758747 28990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:42.758890 28990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "b367f7182fcb4b7f94633abd82a3a2fe"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "15f21e3e4014dfef5a9fad1434795dd0"
server_key_iv: "f7298d3dadf3a13440de2ef88d03cb22"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.758991 28990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:42.781611 28990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:42.784636 28990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:42.784852 28990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:42.785377 28990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:42.786130 28990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:42.786199 28990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.786265 28990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:42.786330 28990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.794607 28990 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:35491
I20260808 18:17:42.794646 29214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:35491 every 8 connection(s)
I20260808 18:17:42.795418 28990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:42.804136 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28990
I20260808 18:17:42.804256 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:17:42.804541 27723 external_mini_cluster.cc:1428] Setting key 3fd834146a3ef5c570b5873e1e5377fa
I20260808 18:17:42.808283 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:42.796972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:55147 (local address 127.27.18.254:43011)
0808 18:17:42.797122 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:42.797127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:42.797854 (+ 727us) server_negotiation.cc:407] Connection header received
0808 18:17:42.799040 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:42.799043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:42.799090 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:42.799163 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:42.800420 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.800944 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.802014 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.802216 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.802289 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:42.804724 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:42.804728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:42.804730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:42.804751 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:42.806234 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:42.806719 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:42.806722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:42.806724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:42.806812 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:42.807145 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:42.807149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:42.807151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:42.807355 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:42.807524 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:42.808002 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 18:17:42.808119 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:17:42.808979 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:42.797303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43011 (local address 127.27.18.194:55147)
0808 18:17:42.797715 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:42.797754 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:42.798861 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:42.799324 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:42.799330 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:42.799834 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:42.800263 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:42.800275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.801047 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.801049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:42.801896 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:42.801905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:42.802342 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:42.802345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:42.802482 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:42.802929 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:42.802945 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:42.804549 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:42.806350 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:42.806356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:42.806368 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:42.806609 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:42.806906 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:42.806909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:42.806911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:42.807047 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:42.807522 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:42.807529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:42.807756 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 18:17:42.807944 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 18:17:42.808229 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":114,"threads_started":1}
I20260808 18:17:42.810127 29216 heartbeater.cc:347] Connected to a master server at 127.27.18.254:43011
I20260808 18:17:42.810374 29216 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:42.810949 29216 heartbeater.cc:511] Master 127.27.18.254:43011 requested a full tablet report, sending...
I20260808 18:17:42.812080 28650 ts_manager.cc:194] Registered new tserver with Master: b367f7182fcb4b7f94633abd82a3a2fe (127.27.18.194:35491)
I20260808 18:17:42.812634 28650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:55147
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:17:42.831729 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:43011
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:42.935113 29222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:42.935423 29222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:42.935503 29222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:42.939572 29222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:42.939640 29222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:42.939718 29222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:17:42.944736 29222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:43011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:42.945631 29222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:42.946533 29222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:17:42.956449 29222 server_base.cc:1034] running on GCE node
W20260808 18:17:42.956339 29241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.956373 29236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:42.956357 29233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:42.957031 29222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:42.957580 29222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:42.958758 29222 hybrid_clock.cc:648] HybridClock initialized: now 1786213062958741 us; error 28 us; skew 500 ppm
Aug 08 18:17:42 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:42.961457 29222 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:17:42.962240 29222 webserver.cc:460] Webserver started at http://127.27.18.195:35769/ using document root <none> and password file <none>
I20260808 18:17:42.962781 29222 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:42.962845 29222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:42.962997 29222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:42.964875 29222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "29c54ae3bd254363aac57529a6600bdd"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "65fef9c87de5dcd762543bbf8f10a6a8"
server_key_iv: "7ca4148dd73ab356e8683ecc191b257a"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.965399 29222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "29c54ae3bd254363aac57529a6600bdd"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "65fef9c87de5dcd762543bbf8f10a6a8"
server_key_iv: "7ca4148dd73ab356e8683ecc191b257a"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.968848 29222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:17:42.970997 29252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.972158 29222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:42.972275 29222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "29c54ae3bd254363aac57529a6600bdd"
format_stamp: "Formatted at 2026-08-08 18:17:42 on dist-test-slave-x4qh"
server_key: "65fef9c87de5dcd762543bbf8f10a6a8"
server_key_iv: "7ca4148dd73ab356e8683ecc191b257a"
server_key_version: "encryptionkey@0"
I20260808 18:17:42.972381 29222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:42.993913 29222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:42.996383 29222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:42.996577 29222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:42.997094 29222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:42.997846 29222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:42.997888 29222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:42.997951 29222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:42.997991 29222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:43.006845 29222 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:38103
I20260808 18:17:43.006901 29448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:38103 every 8 connection(s)
I20260808 18:17:43.007686 29222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:43.016754 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29222
I20260808 18:17:43.016846 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:17:43.017113 27723 external_mini_cluster.cc:1428] Setting key 4fd4d3e257cff6fd487e1195a53a8c82
I20260808 18:17:43.020052 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.009361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:35583 (local address 127.27.18.254:43011)
0808 18:17:43.009523 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.009528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.010288 (+ 760us) server_negotiation.cc:407] Connection header received
0808 18:17:43.011495 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.011498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.011543 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.011615 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:43.012646 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.013166 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.014218 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.014431 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.014499 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.016594 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:43.016597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:43.016599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:43.016621 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:43.018027 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.018553 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.018558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.018560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.018641 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.018980 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.018985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.018987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.019159 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:43.019267 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.019795 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.019904 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:17:43.020686 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.009705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43011 (local address 127.27.18.195:35583)
0808 18:17:43.010142 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:43.010181 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:43.011258 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:43.011763 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:43.011768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:43.012078 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:43.012503 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.012515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.013270 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.013272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.014116 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.014124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.014545 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.014547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.014675 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.015095 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:43.015110 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:43.016454 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:43.018143 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:43.018148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:43.018160 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:43.018444 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:43.018736 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:43.018740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:43.018742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:43.018882 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:43.019267 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:43.019273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:43.019573 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 18:17:43.019754 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 18:17:43.020027 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":123,"threads_started":1}
I20260808 18:17:43.021596 29451 heartbeater.cc:347] Connected to a master server at 127.27.18.254:43011
I20260808 18:17:43.021806 29451 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:43.022229 29451 heartbeater.cc:511] Master 127.27.18.254:43011 requested a full tablet report, sending...
I20260808 18:17:43.023136 28650 ts_manager.cc:194] Registered new tserver with Master: 29c54ae3bd254363aac57529a6600bdd (127.27.18.195:38103)
I20260808 18:17:43.023746 28650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:35583
I20260808 18:17:43.030560 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:17:43.040494 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.034414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.27.18.254:43011)
0808 18:17:43.034514 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.034517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.034669 (+ 152us) server_negotiation.cc:407] Connection header received
0808 18:17:43.034752 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.034754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.034785 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.034846 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:43.035484 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.035920 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.036612 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.036791 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.036858 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.037923 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:43.037926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:43.037927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:43.037947 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:43.039090 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.039578 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.039581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.039582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.039652 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.039871 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.039873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.039874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.040040 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:43.040135 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.040302 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.040384 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:17:43.045428 28650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:17:43.048657 28650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:43.064060 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.058113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:42605 (local address 127.0.0.1:43686)
0808 18:17:43.058474 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:43.058523 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:43.058638 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:43.059633 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:43.059638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:43.059660 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:43.059823 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.059832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.061341 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.061344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.062522 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.062531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.063520 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.063524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.063629 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.063673 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 18:17:43.063785 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 18:17:43.063901 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":107,"spinlock_wait_cycles":1792,"thread_start_us":94,"threads_started":1}
I20260808 18:17:43.064292 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.058716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.27.18.195:38103)
0808 18:17:43.059438 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.059449 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.059469 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:17:43.059541 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.059548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.059703 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.059881 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:43.060082 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.061199 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.062800 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.063677 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.063802 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.063922 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.064072 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.064176 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":134,"threads_started":1}
I20260808 18:17:43.064225 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.058283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:38103 (local address 127.0.0.1:57438)
0808 18:17:43.059331 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:43.059344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:43.059453 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:43.059869 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:43.059872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:43.059888 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:43.059974 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.059980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.061380 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.061383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.062658 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.062669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.063834 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.063838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.063928 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.063962 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:17:43.064038 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:17:43.064100 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":161,"thread_start_us":90,"threads_started":1}
I20260808 18:17:43.065194 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.059083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:35491 (local address 127.0.0.1:38190)
0808 18:17:43.059622 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:43.059633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:43.059715 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:43.060118 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:43.060121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:43.060136 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:43.060205 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.060210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.061124 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.061129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.063243 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.063254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.064774 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.064777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.064844 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.064863 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:17:43.064964 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:17:43.065040 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":694,"spinlock_wait_cycles":384,"thread_start_us":85,"threads_started":1}
I20260808 18:17:43.065229 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.059238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.27.18.194:35491)
0808 18:17:43.059692 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.059702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.059722 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:17:43.059826 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.059850 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.060014 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.060145 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:43.060312 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.060995 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.063417 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.064228 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.064469 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.064648 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.065000 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.065090 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":170,"threads_started":1}
I20260808 18:17:43.067301 29340 tablet_service.cc:1467] Processing CreateTablet for tablet c049c5fdb36f4c199d7c19ea2a9dac95 (DEFAULT_TABLE table=test-table [id=46732ca0717d4ee08f73391c75a70f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:43.068005 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.058257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.27.18.193:42605)
0808 18:17:43.059133 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.059151 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.059172 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:17:43.059246 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.059251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.059469 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.059953 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:43.060104 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.061204 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.062648 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.063288 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.063502 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.063701 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.067751 (+ 4050us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.067846 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":103,"threads_started":1}
I20260808 18:17:43.067875 29107 tablet_service.cc:1467] Processing CreateTablet for tablet c049c5fdb36f4c199d7c19ea2a9dac95 (DEFAULT_TABLE table=test-table [id=46732ca0717d4ee08f73391c75a70f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:43.068506 29340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c049c5fdb36f4c199d7c19ea2a9dac95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:43.068835 29107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c049c5fdb36f4c199d7c19ea2a9dac95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:43.070327 28860 tablet_service.cc:1467] Processing CreateTablet for tablet c049c5fdb36f4c199d7c19ea2a9dac95 (DEFAULT_TABLE table=test-table [id=46732ca0717d4ee08f73391c75a70f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:43.071220 28860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c049c5fdb36f4c199d7c19ea2a9dac95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:43.074874 29477 tablet_bootstrap.cc:492] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe: Bootstrap starting.
I20260808 18:17:43.076138 29478 tablet_bootstrap.cc:492] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd: Bootstrap starting.
I20260808 18:17:43.077966 29477 tablet_bootstrap.cc:654] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:43.078009 29479 tablet_bootstrap.cc:492] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e: Bootstrap starting.
I20260808 18:17:43.078194 29478 tablet_bootstrap.cc:654] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:43.078905 29477 log.cc:824] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:43.079484 29478 log.cc:824] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:43.080219 29479 tablet_bootstrap.cc:654] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:43.081080 29477 tablet_bootstrap.cc:492] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe: No bootstrap required, opened a new log
I20260808 18:17:43.081301 29477 ts_tablet_manager.cc:1397] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 18:17:43.081343 29478 tablet_bootstrap.cc:492] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd: No bootstrap required, opened a new log
I20260808 18:17:43.081486 29479 log.cc:824] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:43.081543 29478 ts_tablet_manager.cc:1397] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 18:17:43.083492 29479 tablet_bootstrap.cc:492] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e: No bootstrap required, opened a new log
I20260808 18:17:43.083673 29479 ts_tablet_manager.cc:1397] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:17:43.084545 29478 raft_consensus.cc:348] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.084893 29478 raft_consensus.cc:374] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:43.084988 29478 raft_consensus.cc:729] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c54ae3bd254363aac57529a6600bdd, State: Initialized, Role: FOLLOWER
I20260808 18:17:43.085209 29477 raft_consensus.cc:348] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.085486 29477 raft_consensus.cc:374] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:43.085417 29478 consensus_queue.cc:260] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.085628 29477 raft_consensus.cc:729] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b367f7182fcb4b7f94633abd82a3a2fe, State: Initialized, Role: FOLLOWER
I20260808 18:17:43.086050 29477 consensus_queue.cc:260] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.086388 29478 ts_tablet_manager.cc:1428] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 18:17:43.086861 29451 heartbeater.cc:503] Master 127.27.18.254:43011 was elected leader, sending a full tablet report...
I20260808 18:17:43.086997 29477 ts_tablet_manager.cc:1428] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:17:43.087359 29216 heartbeater.cc:503] Master 127.27.18.254:43011 was elected leader, sending a full tablet report...
I20260808 18:17:43.087405 29479 raft_consensus.cc:348] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.087653 29479 raft_consensus.cc:374] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:43.087720 29479 raft_consensus.cc:729] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20bad0ca1b3446b49d80f5a171eddc6e, State: Initialized, Role: FOLLOWER
I20260808 18:17:43.087845 29485 raft_consensus.cc:482] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:43.088001 29485 raft_consensus.cc:504] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.088141 29479 consensus_queue.cc:260] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.089051 29479 ts_tablet_manager.cc:1428] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:17:43.089447 28984 heartbeater.cc:503] Master 127.27.18.254:43011 was elected leader, sending a full tablet report...
I20260808 18:17:43.090022 29485 leader_election.cc:288] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20bad0ca1b3446b49d80f5a171eddc6e (127.27.18.193:42605), b367f7182fcb4b7f94633abd82a3a2fe (127.27.18.194:35491)
I20260808 18:17:43.093317 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.089248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:42605 (local address 127.27.18.195:33573)
0808 18:17:43.089387 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:43.089404 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:43.089490 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:43.089832 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:43.089835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:43.089853 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:43.089950 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.089956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.090992 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.090998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.092059 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.092067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.092838 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.092843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.092990 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.093015 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:17:43.093094 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:17:43.093166 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260808 18:17:43.094024 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.089513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:33573 (local address 127.27.18.193:42605)
0808 18:17:43.089629 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.089634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.089649 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:17:43.089691 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.089694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.089731 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.089817 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:43.090102 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.090738 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.092170 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.092587 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.092704 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.092750 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.093865 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.093943 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:17:43.094594 28893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c049c5fdb36f4c199d7c19ea2a9dac95" candidate_uuid: "29c54ae3bd254363aac57529a6600bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" is_pre_election: true
I20260808 18:17:43.095018 28893 raft_consensus.cc:2454] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c54ae3bd254363aac57529a6600bdd in term 0.
I20260808 18:17:43.095552 29256 leader_election.cc:302] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20bad0ca1b3446b49d80f5a171eddc6e, 29c54ae3bd254363aac57529a6600bdd; no voters:
I20260808 18:17:43.095785 29485 raft_consensus.cc:2781] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:43.095885 29485 raft_consensus.cc:482] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:43.095942 29485 raft_consensus.cc:3037] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:43.095963 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.090734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:37897 (local address 127.27.18.194:35491)
0808 18:17:43.090838 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.090846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.090881 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:17:43.091049 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.091053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.091103 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.091154 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:43.091639 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.092236 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.094900 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.095362 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.095526 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.095576 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.095788 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.095851 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:17:43.096035 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.090396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:35491 (local address 127.27.18.195:37897)
0808 18:17:43.090805 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:43.090819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:43.090922 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:43.091291 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:43.091294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:43.091311 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:43.091467 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.091475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.092386 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.092389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.094686 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:43.094697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.095526 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.095530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:43.095640 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.095693 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 18:17:43.095775 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:17:43.095841 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":1300,"thread_start_us":88,"threads_started":1}
I20260808 18:17:43.096589 29142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c049c5fdb36f4c199d7c19ea2a9dac95" candidate_uuid: "29c54ae3bd254363aac57529a6600bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" is_pre_election: true
I20260808 18:17:43.096761 29142 raft_consensus.cc:2454] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29c54ae3bd254363aac57529a6600bdd in term 0.
I20260808 18:17:43.096877 29485 raft_consensus.cc:504] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.097220 29485 leader_election.cc:288] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [CANDIDATE]: Term 1 election: Requested vote from peers 20bad0ca1b3446b49d80f5a171eddc6e (127.27.18.193:42605), b367f7182fcb4b7f94633abd82a3a2fe (127.27.18.194:35491)
I20260808 18:17:43.097522 28893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c049c5fdb36f4c199d7c19ea2a9dac95" candidate_uuid: "29c54ae3bd254363aac57529a6600bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20bad0ca1b3446b49d80f5a171eddc6e"
I20260808 18:17:43.097595 29142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c049c5fdb36f4c199d7c19ea2a9dac95" candidate_uuid: "29c54ae3bd254363aac57529a6600bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b367f7182fcb4b7f94633abd82a3a2fe"
I20260808 18:17:43.097635 28893 raft_consensus.cc:3037] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:43.097754 29142 raft_consensus.cc:3037] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:43.098520 28893 raft_consensus.cc:2454] T c049c5fdb36f4c199d7c19ea2a9dac95 P 20bad0ca1b3446b49d80f5a171eddc6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c54ae3bd254363aac57529a6600bdd in term 1.
I20260808 18:17:43.098790 29142 raft_consensus.cc:2454] T c049c5fdb36f4c199d7c19ea2a9dac95 P b367f7182fcb4b7f94633abd82a3a2fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29c54ae3bd254363aac57529a6600bdd in term 1.
I20260808 18:17:43.098815 29256 leader_election.cc:302] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20bad0ca1b3446b49d80f5a171eddc6e, 29c54ae3bd254363aac57529a6600bdd; no voters:
I20260808 18:17:43.099004 29485 raft_consensus.cc:2781] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:43.099213 29485 raft_consensus.cc:686] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [term 1 LEADER]: Becoming Leader. State: Replica: 29c54ae3bd254363aac57529a6600bdd, State: Running, Role: LEADER
I20260808 18:17:43.099509 29485 consensus_queue.cc:237] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } }
I20260808 18:17:43.102329 28648 catalog_manager.cc:5495] T c049c5fdb36f4c199d7c19ea2a9dac95 P 29c54ae3bd254363aac57529a6600bdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 29c54ae3bd254363aac57529a6600bdd (127.27.18.195). New cstate: current_term: 1 leader_uuid: "29c54ae3bd254363aac57529a6600bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20bad0ca1b3446b49d80f5a171eddc6e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 42605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c54ae3bd254363aac57529a6600bdd" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b367f7182fcb4b7f94633abd82a3a2fe" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35491 } health_report { overall_health: UNKNOWN } } }
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:17:43.114300 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.106489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.27.18.193:42605)
0808 18:17:43.106625 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.106629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.106800 (+ 171us) server_negotiation.cc:407] Connection header received
0808 18:17:43.106882 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.106884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.106919 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.106985 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:43.107677 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.108116 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.108904 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.109087 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.109162 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.111304 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:43.111312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:43.111322 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:43.111343 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:43.112908 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.113259 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.113264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.113265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.113351 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.113568 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.113570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.113571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.113776 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:43.113914 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.114038 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.114115 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:17:43.140302 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.132992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.27.18.193:42605)
0808 18:17:43.133108 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.133112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.133268 (+ 156us) server_negotiation.cc:407] Connection header received
0808 18:17:43.133361 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.133363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.133396 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.133469 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:43.134052 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.134490 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.135226 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.135515 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.135590 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.137482 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:43.137484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:43.137486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:43.137504 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:43.138733 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.139090 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.139095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.139096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.139178 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.139516 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.139519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.139521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.139670 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:43.139801 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.140050 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.140160 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[28574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[28574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:17:43.163764 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:43.156255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.27.18.193:42605)
0808 18:17:43.156403 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:43.156407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:43.156551 (+ 144us) server_negotiation.cc:407] Connection header received
0808 18:17:43.156678 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:43.156681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:43.156721 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:43.156796 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:43.157418 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.157829 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.158553 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:43.158732 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:43.158823 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:43.160901 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:43.160904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:43.160905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:43.160924 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:43.162294 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.162717 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.162720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.162721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.162788 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:43.163019 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:43.163021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:43.163022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:43.163149 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:43.163235 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:43.163505 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 18:17:43.163583 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 18:17:43.165132 28824 tablet_service.cc:2908] Rejecting scan request for tablet c049c5fdb36f4c199d7c19ea2a9dac95: Uninitialized: clean time has not yet been initialized
I20260808 18:17:43.167003 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28754
I20260808 18:17:43.180274 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28990
I20260808 18:17:43.189985 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29222
I20260808 18:17:43.201608 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28598
2026-08-08T18:17:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2369 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[29516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[29516](info): set up 2 sockets
Aug 08 18:17:43 dist-test-slave-x4qh krb5kdc[29516](info): commencing operation
krb5kdc: starting...
W20260808 18:17:44.388782 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:44 dist-test-slave-x4qh krb5kdc[29516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:44.509841 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:37267
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:37267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:44.618125 29538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:44.618366 29538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:44.618402 29538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:44.622303 29538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:44.622349 29538 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:17:44.622365 29538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:44.622383 29538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:44.622398 29538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:44.622412 29538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:44.627426 29538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:37267
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:37267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:44.628427 29538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:44.629427 29538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:17:44.637851 29538 server_base.cc:1034] running on GCE node
W20260808 18:17:44.637862 29549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:44.637893 29548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:44.637876 29553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:44.638618 29538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:44.639703 29538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:44.640879 29538 hybrid_clock.cc:648] HybridClock initialized: now 1786213064640853 us; error 40 us; skew 500 ppm
Aug 08 18:17:44 dist-test-slave-x4qh krb5kdc[29516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213064, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:44.643520 29538 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:44.644351 29538 webserver.cc:460] Webserver started at http://127.27.18.254:40529/ using document root <none> and password file <none>
I20260808 18:17:44.644862 29538 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:44.644920 29538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:44.645076 29538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:44.646977 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "5a0a524e3e6647beb31be7838bfb1c56"
format_stamp: "Formatted at 2026-08-08 18:17:44 on dist-test-slave-x4qh"
server_key: "1760e26cd9c754ba5936785fafc83450"
server_key_iv: "4913b1073648a1878762597629026b7e"
server_key_version: "encryptionkey@0"
I20260808 18:17:44.647550 29538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "5a0a524e3e6647beb31be7838bfb1c56"
format_stamp: "Formatted at 2026-08-08 18:17:44 on dist-test-slave-x4qh"
server_key: "1760e26cd9c754ba5936785fafc83450"
server_key_iv: "4913b1073648a1878762597629026b7e"
server_key_version: "encryptionkey@0"
I20260808 18:17:44.650996 29538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:17:44.653306 29562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:44.654453 29538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:44.654580 29538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5a0a524e3e6647beb31be7838bfb1c56"
format_stamp: "Formatted at 2026-08-08 18:17:44 on dist-test-slave-x4qh"
server_key: "1760e26cd9c754ba5936785fafc83450"
server_key_iv: "4913b1073648a1878762597629026b7e"
server_key_version: "encryptionkey@0"
I20260808 18:17:44.654681 29538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:44.682082 29538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:44.684834 29538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:44.685032 29538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:44.693403 29538 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:37267
I20260808 18:17:44.693435 29663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:37267 every 8 connection(s)
I20260808 18:17:44.694274 29538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:44.695565 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29538
I20260808 18:17:44.695681 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:44.695956 27723 external_mini_cluster.cc:1428] Setting key 3d4ac846f3ed7e90731c527585e21e7a
I20260808 18:17:44.697350 29664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:44.702575 29664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: Bootstrap starting.
Aug 08 18:17:44 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:44.705184 29664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:44.706521 29664 log.cc:824] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:44.708660 29664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: No bootstrap required, opened a new log
I20260808 18:17:44.709472 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:44.697365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.27.18.254:37267)
0808 18:17:44.697786 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:44.697800 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:44.697841 (+ 41us) server_negotiation.cc:407] Connection header received
0808 18:17:44.698445 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:44.698459 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:44.698742 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:44.699068 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:44.699941 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.700893 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.701747 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.702212 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.702374 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:44.705014 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:44.705019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:44.705030 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:44.705055 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:44.706776 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:44.707246 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:44.707252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:44.707259 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:44.707448 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:44.707729 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:44.707731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:44.707732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:44.708095 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:44.708228 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:44.708540 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 18:17:44.708841 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":133,"threads_started":1}
I20260808 18:17:44.711971 29664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0a524e3e6647beb31be7838bfb1c56" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 37267 } }
I20260808 18:17:44.712172 29664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:44.712263 29664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0a524e3e6647beb31be7838bfb1c56, State: Initialized, Role: FOLLOWER
I20260808 18:17:44.712703 29664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0a524e3e6647beb31be7838bfb1c56" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 37267 } }
I20260808 18:17:44.712852 29664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:44.712917 29664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:44.713034 29664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:44.713892 29664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0a524e3e6647beb31be7838bfb1c56" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 37267 } }
I20260808 18:17:44.714205 29664 leader_election.cc:302] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0a524e3e6647beb31be7838bfb1c56; no voters:
I20260808 18:17:44.714510 29664 leader_election.cc:288] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:44.714619 29671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:44.714867 29671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [term 1 LEADER]: Becoming Leader. State: Replica: 5a0a524e3e6647beb31be7838bfb1c56, State: Running, Role: LEADER
I20260808 18:17:44.715224 29671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0a524e3e6647beb31be7838bfb1c56" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 37267 } }
I20260808 18:17:44.715739 29664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:44.717140 29672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0a524e3e6647beb31be7838bfb1c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0a524e3e6647beb31be7838bfb1c56" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 37267 } } }
I20260808 18:17:44.717131 29674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0a524e3e6647beb31be7838bfb1c56. Latest consensus state: current_term: 1 leader_uuid: "5a0a524e3e6647beb31be7838bfb1c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0a524e3e6647beb31be7838bfb1c56" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 37267 } } }
I20260808 18:17:44.717271 29672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:44.717276 29674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:44.718515 29696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:44.721371 29696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:44.726564 29696 catalog_manager.cc:1347] Generated new cluster ID: 56b48f02af47484399ba363d8833e7a6
I20260808 18:17:44.726630 29696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:44.738517 29696 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:44.739364 29696 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:44.746078 29696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: Generated new TSK 0
I20260808 18:17:44.746654 29696 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:17:44.778168 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:37267
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:44.901252 29712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:44.901461 29712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:44.901502 29712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:44.905887 29712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:44.905941 29712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:44.905977 29712 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:17:44.906054 29712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:17:44.910890 29712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:37267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:44.911901 29712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:44.912910 29712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:44.922257 29730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:44.922266 29728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:44.922317 29727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:44.922427 29712 server_base.cc:1034] running on GCE node
I20260808 18:17:44.922925 29712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:44.923542 29712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:44.924726 29712 hybrid_clock.cc:648] HybridClock initialized: now 1786213064924704 us; error 32 us; skew 500 ppm
Aug 08 18:17:44 dist-test-slave-x4qh krb5kdc[29516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213064, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:44.927531 29712 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:17:44.928372 29712 webserver.cc:460] Webserver started at http://127.27.18.193:43061/ using document root <none> and password file <none>
I20260808 18:17:44.928861 29712 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:44.928920 29712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:44.929072 29712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:44.930918 29712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "461033e6b2174cd48c156ea67c33523d"
format_stamp: "Formatted at 2026-08-08 18:17:44 on dist-test-slave-x4qh"
server_key: "0737f0eae0c6b5e8daa1f26c9ec92088"
server_key_iv: "3d0988a34060d43d022dfdd27e9b4719"
server_key_version: "encryptionkey@0"
I20260808 18:17:44.931458 29712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "461033e6b2174cd48c156ea67c33523d"
format_stamp: "Formatted at 2026-08-08 18:17:44 on dist-test-slave-x4qh"
server_key: "0737f0eae0c6b5e8daa1f26c9ec92088"
server_key_iv: "3d0988a34060d43d022dfdd27e9b4719"
server_key_version: "encryptionkey@0"
I20260808 18:17:44.934794 29712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:17:44.937136 29743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:44.938302 29712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:17:44.938393 29712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "461033e6b2174cd48c156ea67c33523d"
format_stamp: "Formatted at 2026-08-08 18:17:44 on dist-test-slave-x4qh"
server_key: "0737f0eae0c6b5e8daa1f26c9ec92088"
server_key_iv: "3d0988a34060d43d022dfdd27e9b4719"
server_key_version: "encryptionkey@0"
I20260808 18:17:44.938468 29712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:44.966853 29712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:44.969974 29712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:44.970122 29712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:44.971477 29712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:44.971525 29712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:44.971582 29712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:44.971622 29712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:17:44 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213064, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:44.982407 29712 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:36167
I20260808 18:17:44.982882 29938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:36167 every 8 connection(s)
I20260808 18:17:44.983490 29712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:17:44.984789 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29712
I20260808 18:17:44.984922 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:17:44.985198 27723 external_mini_cluster.cc:1428] Setting key 2d1ddac0caec9fc2f08bd846b4e30aa2
I20260808 18:17:44.985539 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:44.972370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:35407 (local address 127.27.18.254:37267)
0808 18:17:44.972528 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:44.972533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:44.973402 (+ 869us) server_negotiation.cc:407] Connection header received
0808 18:17:44.974683 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:44.974689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:44.974754 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:44.974846 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:44.975999 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.976592 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.977837 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.978083 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.978195 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:44.981709 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:44.981712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:44.981714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:44.981737 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:44.983326 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:44.983921 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:44.983925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:44.983928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:44.984016 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:44.984418 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:44.984423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:44.984424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:44.984647 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:44.984754 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:44.985265 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 18:17:44.985394 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:17:44.986301 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:44.972725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.27.18.193:35407)
0808 18:17:44.973213 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:44.973271 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:44.974360 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:44.975005 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:44.975011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:44.975285 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:44.975857 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:44.975873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.976731 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.976734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:44.977716 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:44.977726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.978235 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.978238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:44.978387 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:44.978917 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:44.978933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:44.981539 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:44.983466 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:44.983475 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:44.983485 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:44.983785 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:44.984130 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:44.984135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:44.984139 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:44.984295 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:44.984755 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:44.984759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:44.985028 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 18:17:44.985185 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 18:17:44.985455 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260808 18:17:44.987722 29939 heartbeater.cc:347] Connected to a master server at 127.27.18.254:37267
I20260808 18:17:44.987908 29939 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:44.988488 29939 heartbeater.cc:511] Master 127.27.18.254:37267 requested a full tablet report, sending...
I20260808 18:17:44.989912 29588 ts_manager.cc:194] Registered new tserver with Master: 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167)
I20260808 18:17:44.991510 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:35407
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:17:44.997426 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:44.990624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:41199 (local address 127.27.18.254:37267)
0808 18:17:44.990727 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:44.990731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:44.990956 (+ 225us) server_negotiation.cc:407] Connection header received
0808 18:17:44.991078 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:44.991081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:44.991119 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:44.991213 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:44.991759 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.992309 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.993115 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.993350 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.993452 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:44.994499 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:44.994502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:44.994505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:44.994528 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:44.995903 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:44.996313 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:44.996318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:44.996319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:44.996397 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:44.996682 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:44.996687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:44.996688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:44.996839 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:44.996939 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:44.997199 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 18:17:44.997296 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:17:44.997432 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:44.990579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.27.18.193:41199)
0808 18:17:44.990864 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:44.990876 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:44.990981 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:44.991229 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:44.991232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:44.991472 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:44.991563 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:44.991568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.992428 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.992431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:44.992997 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:44.993004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:44.993467 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:44.993470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:44.993569 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:44.994021 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:44.994051 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:44.994354 (+ 303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:44.996014 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:44.996019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:44.996021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:44.996214 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:44.996490 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:44.996494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:44.996496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:44.996577 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:44.996933 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:44.996936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:44.997025 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 18:17:44.997120 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:17:44.997247 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:17:45.014621 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:37267
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:45.119896 29952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:45.120122 29952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:45.120193 29952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:45.123761 29952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:45.123837 29952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:45.123916 29952 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:17:45.123986 29952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:17:45.128584 29952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:37267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:45.129484 29952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:45.130379 29952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:45.139227 29962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:45.139231 29961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:45.139561 29952 server_base.cc:1034] running on GCE node
W20260808 18:17:45.139243 29966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:45.139914 29952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:45.140467 29952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:45.141647 29952 hybrid_clock.cc:648] HybridClock initialized: now 1786213065141621 us; error 37 us; skew 500 ppm
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:45.144397 29952 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:17:45.145133 29952 webserver.cc:460] Webserver started at http://127.27.18.194:42091/ using document root <none> and password file <none>
I20260808 18:17:45.145629 29952 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:45.145720 29952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:45.145937 29952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:45.148025 29952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "8c9f6a0091684361b7bfe733383819e7"
format_stamp: "Formatted at 2026-08-08 18:17:45 on dist-test-slave-x4qh"
server_key: "36c636b4609fa844bbfe8ab89354943b"
server_key_iv: "563977e9291dc80f3d74e0401658ae41"
server_key_version: "encryptionkey@0"
I20260808 18:17:45.148612 29952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "8c9f6a0091684361b7bfe733383819e7"
format_stamp: "Formatted at 2026-08-08 18:17:45 on dist-test-slave-x4qh"
server_key: "36c636b4609fa844bbfe8ab89354943b"
server_key_iv: "563977e9291dc80f3d74e0401658ae41"
server_key_version: "encryptionkey@0"
I20260808 18:17:45.151968 29952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:17:45.154193 29980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:45.155328 29952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:45.155478 29952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "8c9f6a0091684361b7bfe733383819e7"
format_stamp: "Formatted at 2026-08-08 18:17:45 on dist-test-slave-x4qh"
server_key: "36c636b4609fa844bbfe8ab89354943b"
server_key_iv: "563977e9291dc80f3d74e0401658ae41"
server_key_version: "encryptionkey@0"
I20260808 18:17:45.155586 29952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:45.165397 29952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:45.168318 29952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:45.168509 29952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:45.169818 29952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:45.169890 29952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:45.169965 29952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:45.170018 29952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:45.180908 29952 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:44595
I20260808 18:17:45.180974 30158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:44595 every 8 connection(s)
I20260808 18:17:45.181720 29952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:17:45.184111 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.170603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:55745 (local address 127.27.18.254:37267)
0808 18:17:45.170728 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.170732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.171828 (+ 1096us) server_negotiation.cc:407] Connection header received
0808 18:17:45.172840 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.172844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.172910 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.172971 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:45.174735 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.175643 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.177310 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.177707 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.177846 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.180645 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:45.180650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:45.180653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:45.180682 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:45.182324 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.182842 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.182845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.182846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.182926 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.183194 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.183197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.183198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.183361 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:45.183500 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.183891 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.183999 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:17:45.184854 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.171105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.27.18.194:55745)
0808 18:17:45.171688 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.171727 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.172550 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.173399 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.173410 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.173906 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:45.174562 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.174581 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.175784 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.175788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.177169 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.177182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.177878 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.177884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.178070 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.178622 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:45.178638 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:45.180433 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:45.182456 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.182462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.182474 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.182712 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.182998 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.183000 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.183001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.183118 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.183493 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:45.183498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:45.183691 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.183836 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.184056 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":167,"threads_started":1}
I20260808 18:17:45.185979 30161 heartbeater.cc:347] Connected to a master server at 127.27.18.254:37267
I20260808 18:17:45.186142 30161 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:45.186679 30161 heartbeater.cc:511] Master 127.27.18.254:37267 requested a full tablet report, sending...
I20260808 18:17:45.187971 29589 ts_manager.cc:194] Registered new tserver with Master: 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595)
I20260808 18:17:45.188733 29589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:55745
I20260808 18:17:45.190820 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29952
I20260808 18:17:45.190945 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:17:45.191272 27723 external_mini_cluster.cc:1428] Setting key 1cec1c9e4ab5826e91d4a092b97ebe11
I20260808 18:17:45.195262 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.188149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:36803 (local address 127.27.18.254:37267)
0808 18:17:45.188307 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.188311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.188495 (+ 184us) server_negotiation.cc:407] Connection header received
0808 18:17:45.188575 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.188578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.188616 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.188684 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:45.189213 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.189655 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.190494 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.190700 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.190807 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.191931 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:45.191934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:45.191936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:45.191956 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:45.193572 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.194021 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.194027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.194031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.194121 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.194375 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.194381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.194385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.194661 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:45.194767 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.195088 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.195169 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:17:45.195463 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.188045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.27.18.194:36803)
0808 18:17:45.188415 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.188428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.188502 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.188708 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.188710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.188973 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:45.189104 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.189111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.189819 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.189822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.190365 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.190372 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.190817 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.190821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.190920 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.191473 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:45.191484 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:45.191761 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:45.193691 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.193694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.193695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.193914 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.194213 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.194215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.194216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.194287 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.194780 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:45.194786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:45.194905 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.195020 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.195168 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:17:45.217854 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:37267
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:45.324854 30176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:45.325114 30176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:45.325191 30176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:45.328989 30176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:45.329066 30176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:45.329147 30176 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:17:45.329233 30176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:17:45.334275 30176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:37267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:45.335202 30176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:45.336233 30176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:45.345534 30184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:45.345568 30183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:45.345571 30186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:45.345826 30176 server_base.cc:1034] running on GCE node
I20260808 18:17:45.346216 30176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:45.346729 30176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:45.347890 30176 hybrid_clock.cc:648] HybridClock initialized: now 1786213065347861 us; error 40 us; skew 500 ppm
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:45.350361 30176 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:17:45.351202 30176 webserver.cc:460] Webserver started at http://127.27.18.195:34321/ using document root <none> and password file <none>
I20260808 18:17:45.351787 30176 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:45.351866 30176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:45.352090 30176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:45.353977 30176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "626c770d2a5d4503a28d1b3cf4835cc6"
format_stamp: "Formatted at 2026-08-08 18:17:45 on dist-test-slave-x4qh"
server_key: "0e7756a3596e0f9a785b94f5811e0342"
server_key_iv: "58b2e2c4996b1d268e7d4553ac52caba"
server_key_version: "encryptionkey@0"
I20260808 18:17:45.354524 30176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "626c770d2a5d4503a28d1b3cf4835cc6"
format_stamp: "Formatted at 2026-08-08 18:17:45 on dist-test-slave-x4qh"
server_key: "0e7756a3596e0f9a785b94f5811e0342"
server_key_iv: "58b2e2c4996b1d268e7d4553ac52caba"
server_key_version: "encryptionkey@0"
I20260808 18:17:45.358130 30176 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 18:17:45.360656 30197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:45.361755 30176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:45.361891 30176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "626c770d2a5d4503a28d1b3cf4835cc6"
format_stamp: "Formatted at 2026-08-08 18:17:45 on dist-test-slave-x4qh"
server_key: "0e7756a3596e0f9a785b94f5811e0342"
server_key_iv: "58b2e2c4996b1d268e7d4553ac52caba"
server_key_version: "encryptionkey@0"
I20260808 18:17:45.361996 30176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:45.381342 30176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:45.383882 30176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:45.384073 30176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:45.385408 30176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:45.385486 30176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:45.385562 30176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:45.385607 30176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:45.397054 30176 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42565
I20260808 18:17:45.397099 30375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42565 every 8 connection(s)
I20260808 18:17:45.398015 30176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:17:45.398999 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.386175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:43249 (local address 127.27.18.254:37267)
0808 18:17:45.386345 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.386349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.387090 (+ 741us) server_negotiation.cc:407] Connection header received
0808 18:17:45.388036 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.388043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.388104 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.388199 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:45.389464 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.390095 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.391243 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.391595 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.391736 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.395073 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:45.395077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:45.395080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:45.395105 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:45.396933 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.397520 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.397524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.397526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.397621 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.397943 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.397946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.397948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.398175 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:45.398274 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.398784 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.398891 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:17:45.399771 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.386544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.27.18.195:43249)
0808 18:17:45.386932 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.386974 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.387824 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.388510 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.388520 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.388910 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:45.389285 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.389323 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.390201 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.390206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.391113 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.391122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.391770 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.391774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.391962 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.392479 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:45.392507 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:45.394927 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:45.397075 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.397081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.397094 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.397415 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.397719 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.397722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.397723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.397842 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.398287 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:45.398291 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:45.398551 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.398731 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.399002 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":109,"threads_started":1}
I20260808 18:17:45.400799 30376 heartbeater.cc:347] Connected to a master server at 127.27.18.254:37267
I20260808 18:17:45.401008 30376 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:45.401438 30376 heartbeater.cc:511] Master 127.27.18.254:37267 requested a full tablet report, sending...
I20260808 18:17:45.402401 29588 ts_manager.cc:194] Registered new tserver with Master: 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565)
I20260808 18:17:45.403050 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:43249
I20260808 18:17:45.403529 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30176
I20260808 18:17:45.403661 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:17:45.403962 27723 external_mini_cluster.cc:1428] Setting key 245d7c89734425b05271bedfab342968
I20260808 18:17:45.406108 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:17:45.410579 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.402983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:35865 (local address 127.27.18.254:37267)
0808 18:17:45.403110 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.403116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.403336 (+ 220us) server_negotiation.cc:407] Connection header received
0808 18:17:45.403554 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.403557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.403610 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.403695 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:45.404297 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.404978 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.406034 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.406322 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.406431 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.407490 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:45.407493 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:45.407495 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:45.407514 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:45.408948 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.409469 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.409472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.409473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.409542 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.409849 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.409851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.409852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.409994 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:45.410067 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.410400 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.410484 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:17:45.410781 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.402905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.27.18.195:35865)
0808 18:17:45.403253 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.403265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.403368 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.403828 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.403832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.404028 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:45.404152 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.404160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.405159 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.405162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.405914 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.405926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.406459 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.406462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.406551 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.406953 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:45.406971 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:45.407347 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:45.409061 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.409066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.409070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.409366 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.409635 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.409641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.409644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.409747 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.410094 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:45.410100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:45.410214 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.410348 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.410512 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":99,"threads_started":1}
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:45.432291 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.425127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.27.18.254:37267)
0808 18:17:45.425237 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.425240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.425251 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:17:45.425280 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.425282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.425314 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.425379 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:45.425979 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.426459 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.427221 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.427480 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.427556 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.429558 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:45.429561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:45.429563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:45.429583 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:45.430779 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.431155 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.431158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.431159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.431228 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.431547 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.431550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.431552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.431699 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:45.431831 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.432055 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.432153 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:17:45.434742 29588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:17:45.437587 29588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:45.450834 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.444891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:36167 (local address 127.0.0.1:39124)
0808 18:17:45.445932 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.445976 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.446101 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.446851 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.446855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.446871 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.447089 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.447095 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.448089 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.448093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.449351 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.449361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.450284 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.450288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.450403 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.450461 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.450564 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.450684 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":60,"spinlock_wait_cycles":8448,"thread_start_us":80,"threads_started":1}
I20260808 18:17:45.451031 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.445036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.27.18.193:36167)
0808 18:17:45.446472 (+ 1436us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.446484 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.446499 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:17:45.446554 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.446557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.446676 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.446939 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.447206 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.447978 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.449479 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.450140 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.450249 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.450400 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.450762 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.450880 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1359,"thread_start_us":63,"threads_started":1}
I20260808 18:17:45.450987 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.445009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42565 (local address 127.0.0.1:50788)
0808 18:17:45.445931 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.445975 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.446095 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.446653 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.446659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.446699 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.446920 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.446927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.448193 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.448196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.449556 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.449567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.450541 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.450545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.450645 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.450670 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.450752 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.450846 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":32,"spinlock_wait_cycles":3840,"thread_start_us":80,"threads_started":1}
I20260808 18:17:45.451331 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.445460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.27.18.195:42565)
0808 18:17:45.446212 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.446227 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.446249 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:17:45.446313 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.446317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.446493 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.446937 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.447045 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.448036 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.449710 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.450362 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.450508 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.450670 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.451083 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.451173 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":89,"threads_started":1}
I20260808 18:17:45.451910 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.445779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43556 (local address 127.27.18.194:44595)
0808 18:17:45.446395 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.446405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.446423 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:17:45.446487 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.446491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.446653 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.446824 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.447457 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.448515 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.449892 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.450741 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.450835 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.450977 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.451623 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.451724 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":105,"threads_started":1}
I20260808 18:17:45.452167 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.445458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:44595 (local address 127.0.0.1:43556)
0808 18:17:45.446250 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.446259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.446311 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.446815 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.446820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.446842 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.447304 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.447311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.448659 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.448662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.449770 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.449778 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.451047 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.451050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.451149 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.451171 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.451251 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.451325 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":23,"thread_start_us":92,"threads_started":1}
I20260808 18:17:45.454383 30058 tablet_service.cc:1467] Processing CreateTablet for tablet 1a44554f22084ec0b17de2857adab3c4 (DEFAULT_TABLE table=test-table [id=de74f4c062ad42fbb9a264f40dacf18f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:45.454298 30276 tablet_service.cc:1467] Processing CreateTablet for tablet 1a44554f22084ec0b17de2857adab3c4 (DEFAULT_TABLE table=test-table [id=de74f4c062ad42fbb9a264f40dacf18f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:45.454309 29830 tablet_service.cc:1467] Processing CreateTablet for tablet 1a44554f22084ec0b17de2857adab3c4 (DEFAULT_TABLE table=test-table [id=de74f4c062ad42fbb9a264f40dacf18f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:45.455263 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a44554f22084ec0b17de2857adab3c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.455319 29830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a44554f22084ec0b17de2857adab3c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.455319 30276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a44554f22084ec0b17de2857adab3c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.462551 30414 tablet_bootstrap.cc:492] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Bootstrap starting.
I20260808 18:17:45.463773 30417 tablet_bootstrap.cc:492] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Bootstrap starting.
I20260808 18:17:45.463773 30418 tablet_bootstrap.cc:492] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Bootstrap starting.
I20260808 18:17:45.464594 30414 tablet_bootstrap.cc:654] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.465457 30414 log.cc:824] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:45.465827 30418 tablet_bootstrap.cc:654] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.465831 30417 tablet_bootstrap.cc:654] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.466701 30417 log.cc:824] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:45.466687 30418 log.cc:824] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:45.467798 30414 tablet_bootstrap.cc:492] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: No bootstrap required, opened a new log
I20260808 18:17:45.467991 30414 ts_tablet_manager.cc:1397] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:17:45.468478 30418 tablet_bootstrap.cc:492] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: No bootstrap required, opened a new log
I20260808 18:17:45.468478 30417 tablet_bootstrap.cc:492] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: No bootstrap required, opened a new log
I20260808 18:17:45.468648 30418 ts_tablet_manager.cc:1397] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 18:17:45.468648 30417 ts_tablet_manager.cc:1397] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:17:45.470762 30414 raft_consensus.cc:348] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.470971 30414 raft_consensus.cc:374] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.471040 30414 raft_consensus.cc:729] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9f6a0091684361b7bfe733383819e7, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.471493 30414 consensus_queue.cc:260] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.472079 30161 heartbeater.cc:503] Master 127.27.18.254:37267 was elected leader, sending a full tablet report...
I20260808 18:17:45.472235 30418 raft_consensus.cc:348] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.472235 30417 raft_consensus.cc:348] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.472424 30414 ts_tablet_manager.cc:1428] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 18:17:45.472450 30417 raft_consensus.cc:374] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.472450 30418 raft_consensus.cc:374] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.472497 30417 raft_consensus.cc:729] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626c770d2a5d4503a28d1b3cf4835cc6, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.472497 30418 raft_consensus.cc:729] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461033e6b2174cd48c156ea67c33523d, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.472908 30418 consensus_queue.cc:260] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.472908 30417 consensus_queue.cc:260] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.473835 30418 ts_tablet_manager.cc:1428] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:17:45.473826 30417 ts_tablet_manager.cc:1428] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:17:45.474184 30376 heartbeater.cc:503] Master 127.27.18.254:37267 was elected leader, sending a full tablet report...
I20260808 18:17:45.474210 29939 heartbeater.cc:503] Master 127.27.18.254:37267 was elected leader, sending a full tablet report...
W20260808 18:17:45.484587 29940 tablet.cc:2369] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:17:45.523926 30423 raft_consensus.cc:482] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:45.524117 30423 raft_consensus.cc:504] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.525166 30423 leader_election.cc:288] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167), 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565)
I20260808 18:17:45.529625 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.525407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:36167 (local address 127.27.18.194:45247)
0808 18:17:45.525506 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.525521 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.525609 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.525945 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.525947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.525986 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.526127 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.526134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.527307 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.527311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.528318 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.528327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.529171 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.529174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.529302 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.529327 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.529424 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.529497 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 18:17:45.529625 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.525582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:45247 (local address 127.27.18.193:36167)
0808 18:17:45.525684 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.525708 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.525726 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:17:45.525780 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.525782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.525838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.525927 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.526262 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.527101 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.528448 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.529069 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.529178 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.529236 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.529442 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.529511 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:17:45.530119 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.525407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42565 (local address 127.27.18.194:60051)
0808 18:17:45.525889 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.525905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.526009 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.526280 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.526282 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.526329 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.526926 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.526935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.527988 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.527990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.529004 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.529015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.529685 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.529689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.529792 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.529815 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.529890 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.529957 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":107,"spinlock_wait_cycles":2304,"thread_start_us":136,"threads_started":1}
I20260808 18:17:45.530148 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.525506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:60051 (local address 127.27.18.195:42565)
0808 18:17:45.525639 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.525643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.525988 (+ 345us) server_negotiation.cc:407] Connection header received
0808 18:17:45.526121 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.526124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.526178 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.526260 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.527046 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.527880 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.529156 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.529550 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.529685 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.529733 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.529914 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.530006 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:17:45.530289 29869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44554f22084ec0b17de2857adab3c4" candidate_uuid: "8c9f6a0091684361b7bfe733383819e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461033e6b2174cd48c156ea67c33523d" is_pre_election: true
I20260808 18:17:45.530608 29869 raft_consensus.cc:2454] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9f6a0091684361b7bfe733383819e7 in term 0.
I20260808 18:17:45.530821 30304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44554f22084ec0b17de2857adab3c4" candidate_uuid: "8c9f6a0091684361b7bfe733383819e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" is_pre_election: true
I20260808 18:17:45.531086 30304 raft_consensus.cc:2454] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9f6a0091684361b7bfe733383819e7 in term 0.
I20260808 18:17:45.531175 29985 leader_election.cc:302] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461033e6b2174cd48c156ea67c33523d, 8c9f6a0091684361b7bfe733383819e7; no voters:
I20260808 18:17:45.531523 30423 raft_consensus.cc:2781] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:45.531620 30423 raft_consensus.cc:482] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:45.531698 30423 raft_consensus.cc:3037] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:45.532683 30423 raft_consensus.cc:504] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.533053 30423 leader_election.cc:288] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [CANDIDATE]: Term 1 election: Requested vote from peers 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167), 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565)
I20260808 18:17:45.533378 29869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44554f22084ec0b17de2857adab3c4" candidate_uuid: "8c9f6a0091684361b7bfe733383819e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461033e6b2174cd48c156ea67c33523d"
I20260808 18:17:45.533389 30304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44554f22084ec0b17de2857adab3c4" candidate_uuid: "8c9f6a0091684361b7bfe733383819e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "626c770d2a5d4503a28d1b3cf4835cc6"
I20260808 18:17:45.533504 29869 raft_consensus.cc:3037] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:45.533510 30304 raft_consensus.cc:3037] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:45.534545 29869 raft_consensus.cc:2454] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9f6a0091684361b7bfe733383819e7 in term 1.
I20260808 18:17:45.534807 30304 raft_consensus.cc:2454] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9f6a0091684361b7bfe733383819e7 in term 1.
I20260808 18:17:45.534857 29985 leader_election.cc:302] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461033e6b2174cd48c156ea67c33523d, 8c9f6a0091684361b7bfe733383819e7; no voters:
I20260808 18:17:45.535041 30423 raft_consensus.cc:2781] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:45.535250 30423 raft_consensus.cc:686] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9f6a0091684361b7bfe733383819e7, State: Running, Role: LEADER
I20260808 18:17:45.535584 30423 consensus_queue.cc:237] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } }
I20260808 18:17:45.538650 29589 catalog_manager.cc:5495] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194). New cstate: current_term: 1 leader_uuid: "8c9f6a0091684361b7bfe733383819e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } health_report { overall_health: HEALTHY } } }
I20260808 18:17:45.575778 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.572478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.27.18.194:44595)
0808 18:17:45.572622 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.572625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.572639 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:17:45.572833 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.572835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.572880 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.572958 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:45.573183 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.573642 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.574444 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.574641 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.574736 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.575012 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:45.575411 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:45.575493 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.575570 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.575614 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:17:45.581295 29869 raft_consensus.cc:1261] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Refusing update from remote peer 8c9f6a0091684361b7bfe733383819e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:45.582005 30433 consensus_queue.cc:1035] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:45.583027 30304 raft_consensus.cc:1261] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Refusing update from remote peer 8c9f6a0091684361b7bfe733383819e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:45.583597 30423 consensus_queue.cc:1035] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:45.589808 30442 mvcc.cc:205] Tried to move back new op lower bound from 7316328716613509120 to 7316328716436176896. Current Snapshot: MvccSnapshot[applied={T|T < 7316328716613509120}]
I20260808 18:17:45.590224 30447 mvcc.cc:205] Tried to move back new op lower bound from 7316328716613509120 to 7316328716436176896. Current Snapshot: MvccSnapshot[applied={T|T < 7316328716613509120}]
I20260808 18:17:45.600981 29587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:45.601215 29587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:45.603793 29587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: Sending DeleteTablet for 3 replicas of tablet 1a44554f22084ec0b17de2857adab3c4
I20260808 18:17:45.604571 30058 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a44554f22084ec0b17de2857adab3c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:45 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:43556
I20260808 18:17:45.604574 30276 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a44554f22084ec0b17de2857adab3c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:45 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50788
I20260808 18:17:45.605048 30462 tablet_replica.cc:330] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: stopping tablet replica
I20260808 18:17:45.605067 29830 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a44554f22084ec0b17de2857adab3c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:45 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:39124
I20260808 18:17:45.605437 30462 raft_consensus.cc:2229] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:17:45.605847 30462 raft_consensus.cc:2258] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:45.606181 29589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 18:17:45.606240 30458 tablet_replica.cc:330] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: stopping tablet replica
W20260808 18:17:45.606493 29589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:45.606747 30458 raft_consensus.cc:2229] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:45.607091 30458 raft_consensus.cc:2258] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:45.607240 30463 tablet_replica.cc:330] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: stopping tablet replica
I20260808 18:17:45.607513 30463 raft_consensus.cc:2229] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:45.607767 30463 raft_consensus.cc:2258] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:45.608999 30458 ts_tablet_manager.cc:1905] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:45.609222 30462 ts_tablet_manager.cc:1905] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:45.611231 30463 ts_tablet_manager.cc:1905] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:45.611577 29830 tablet_service.cc:1467] Processing CreateTablet for tablet 445b6c4929c04dbd8b1244e3643d7980 (DEFAULT_TABLE table=test-table [id=97a0079872ed4d07b6721adf8a34848f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:45.611874 29830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445b6c4929c04dbd8b1244e3643d7980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.612228 30458 ts_tablet_manager.cc:1918] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:17:45.612392 30458 log.cc:1196] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/1a44554f22084ec0b17de2857adab3c4
I20260808 18:17:45.612773 30458 ts_tablet_manager.cc:1939] T 1a44554f22084ec0b17de2857adab3c4 P 626c770d2a5d4503a28d1b3cf4835cc6: Deleting consensus metadata
I20260808 18:17:45.613987 30463 ts_tablet_manager.cc:1918] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:17:45.614078 30463 log.cc:1196] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/1a44554f22084ec0b17de2857adab3c4
I20260808 18:17:45.614100 30418 tablet_bootstrap.cc:492] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Bootstrap starting.
I20260808 18:17:45.614437 30463 ts_tablet_manager.cc:1939] T 1a44554f22084ec0b17de2857adab3c4 P 461033e6b2174cd48c156ea67c33523d: Deleting consensus metadata
I20260808 18:17:45.614653 29566 catalog_manager.cc:4841] TS 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565): tablet 1a44554f22084ec0b17de2857adab3c4 (table test-table [id=de74f4c062ad42fbb9a264f40dacf18f]) successfully deleted
I20260808 18:17:45.615020 30418 tablet_bootstrap.cc:654] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.615530 30058 tablet_service.cc:1467] Processing CreateTablet for tablet 445b6c4929c04dbd8b1244e3643d7980 (DEFAULT_TABLE table=test-table [id=97a0079872ed4d07b6721adf8a34848f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:45.615880 29569 catalog_manager.cc:4841] TS 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167): tablet 1a44554f22084ec0b17de2857adab3c4 (table test-table [id=de74f4c062ad42fbb9a264f40dacf18f]) successfully deleted
I20260808 18:17:45.616530 30276 tablet_service.cc:1467] Processing CreateTablet for tablet 445b6c4929c04dbd8b1244e3643d7980 (DEFAULT_TABLE table=test-table [id=97a0079872ed4d07b6721adf8a34848f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:45.616770 30276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445b6c4929c04dbd8b1244e3643d7980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.618805 30417 tablet_bootstrap.cc:492] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Bootstrap starting.
I20260808 18:17:45.619030 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445b6c4929c04dbd8b1244e3643d7980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.619784 30417 tablet_bootstrap.cc:654] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.620957 30417 tablet_bootstrap.cc:492] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: No bootstrap required, opened a new log
I20260808 18:17:45.621027 30417 ts_tablet_manager.cc:1397] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 18:17:45.621459 30462 ts_tablet_manager.cc:1918] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:17:45.621487 30414 tablet_bootstrap.cc:492] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Bootstrap starting.
I20260808 18:17:45.621564 30462 log.cc:1196] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/1a44554f22084ec0b17de2857adab3c4
I20260808 18:17:45.621549 30417 raft_consensus.cc:348] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.621659 30417 raft_consensus.cc:374] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.621694 30417 raft_consensus.cc:729] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626c770d2a5d4503a28d1b3cf4835cc6, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.621997 30462 ts_tablet_manager.cc:1939] T 1a44554f22084ec0b17de2857adab3c4 P 8c9f6a0091684361b7bfe733383819e7: Deleting consensus metadata
I20260808 18:17:45.621937 30417 consensus_queue.cc:260] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.622249 30417 ts_tablet_manager.cc:1428] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 18:17:45.622414 30414 tablet_bootstrap.cc:654] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.623111 29567 catalog_manager.cc:4841] TS 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595): tablet 1a44554f22084ec0b17de2857adab3c4 (table test-table [id=de74f4c062ad42fbb9a264f40dacf18f]) successfully deleted
I20260808 18:17:45.624307 30418 tablet_bootstrap.cc:492] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: No bootstrap required, opened a new log
I20260808 18:17:45.624402 30414 tablet_bootstrap.cc:492] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: No bootstrap required, opened a new log
I20260808 18:17:45.624418 30418 ts_tablet_manager.cc:1397] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260808 18:17:45.624486 30414 ts_tablet_manager.cc:1397] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:17:45.624889 30418 raft_consensus.cc:348] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.624889 30414 raft_consensus.cc:348] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.624995 30418 raft_consensus.cc:374] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.624995 30414 raft_consensus.cc:374] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.625034 30414 raft_consensus.cc:729] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9f6a0091684361b7bfe733383819e7, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.625034 30418 raft_consensus.cc:729] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461033e6b2174cd48c156ea67c33523d, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.625186 30414 consensus_queue.cc:260] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.625252 30418 consensus_queue.cc:260] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.625516 30418 ts_tablet_manager.cc:1428] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:17:45.626737 30414 ts_tablet_manager.cc:1428] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260808 18:17:45.649147 30377 tablet.cc:2369] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:17:45.682825 30164 tablet.cc:2369] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:17:45.781704 30431 raft_consensus.cc:482] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:45.781839 30431 raft_consensus.cc:504] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.783077 30431 leader_election.cc:288] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167), 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595)
I20260808 18:17:45.786770 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.783520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:46523 (local address 127.27.18.194:44595)
0808 18:17:45.783653 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.783660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.783677 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:17:45.783740 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.783742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.783782 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.783870 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.784183 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.784698 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.785785 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.786234 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.786353 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.786407 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.786592 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.786657 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:17:45.786770 30209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.783325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:44595 (local address 127.27.18.195:46523)
0808 18:17:45.783499 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.783516 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.783618 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.783885 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.783888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.783909 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.784043 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.784049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.784815 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.784820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.785637 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.785656 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.786353 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.786356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.786459 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.786480 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.786573 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.786640 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 18:17:45.787036 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.783460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:34133 (local address 127.27.18.193:36167)
0808 18:17:45.783615 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.783620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.783975 (+ 355us) server_negotiation.cc:407] Connection header received
0808 18:17:45.784132 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.784135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.784187 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.784267 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:45.784560 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.785064 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.786089 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.786516 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.786642 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.786687 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.786871 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.786934 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:17:45.787057 30470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.783325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:36167 (local address 127.27.18.195:34133)
0808 18:17:45.783874 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.783886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.784014 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.784277 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.784279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.784295 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:45.784451 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.784458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.785173 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.785178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.785967 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.785975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.786644 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.786648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.786740 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.786760 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.786855 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.786918 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":246,"thread_start_us":104,"threads_started":1}
I20260808 18:17:45.787384 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b6c4929c04dbd8b1244e3643d7980" candidate_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9f6a0091684361b7bfe733383819e7" is_pre_election: true
I20260808 18:17:45.787544 29869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b6c4929c04dbd8b1244e3643d7980" candidate_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461033e6b2174cd48c156ea67c33523d" is_pre_election: true
I20260808 18:17:45.787611 30091 raft_consensus.cc:2454] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 626c770d2a5d4503a28d1b3cf4835cc6 in term 0.
I20260808 18:17:45.787678 29869 raft_consensus.cc:2454] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 626c770d2a5d4503a28d1b3cf4835cc6 in term 0.
I20260808 18:17:45.788002 30203 leader_election.cc:302] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 461033e6b2174cd48c156ea67c33523d, 626c770d2a5d4503a28d1b3cf4835cc6, 8c9f6a0091684361b7bfe733383819e7; no voters:
I20260808 18:17:45.788275 30431 raft_consensus.cc:2781] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:45.788362 30431 raft_consensus.cc:482] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:45.788396 30431 raft_consensus.cc:3037] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:45.789424 30431 raft_consensus.cc:504] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.789746 30431 leader_election.cc:288] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [CANDIDATE]: Term 1 election: Requested vote from peers 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167), 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595)
I20260808 18:17:45.790103 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b6c4929c04dbd8b1244e3643d7980" candidate_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9f6a0091684361b7bfe733383819e7"
I20260808 18:17:45.790132 29869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b6c4929c04dbd8b1244e3643d7980" candidate_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461033e6b2174cd48c156ea67c33523d"
I20260808 18:17:45.790241 30091 raft_consensus.cc:3037] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:45.790239 29869 raft_consensus.cc:3037] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:45.791306 29869 raft_consensus.cc:2454] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 626c770d2a5d4503a28d1b3cf4835cc6 in term 1.
I20260808 18:17:45.791314 30091 raft_consensus.cc:2454] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 626c770d2a5d4503a28d1b3cf4835cc6 in term 1.
I20260808 18:17:45.791635 30202 leader_election.cc:302] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 461033e6b2174cd48c156ea67c33523d, 626c770d2a5d4503a28d1b3cf4835cc6, 8c9f6a0091684361b7bfe733383819e7; no voters:
I20260808 18:17:45.791823 30431 raft_consensus.cc:2781] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:45.792052 30431 raft_consensus.cc:686] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 LEADER]: Becoming Leader. State: Replica: 626c770d2a5d4503a28d1b3cf4835cc6, State: Running, Role: LEADER
I20260808 18:17:45.792280 30431 consensus_queue.cc:237] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.794353 29589 catalog_manager.cc:5495] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } health_report { overall_health: HEALTHY } } }
Aug 08 18:17:45 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213064, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:45.841092 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.832902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.27.18.254:37267)
0808 18:17:45.833036 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:45.833041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:45.833170 (+ 129us) server_negotiation.cc:407] Connection header received
0808 18:17:45.833319 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:45.833321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:45.833365 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:45.833447 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:45.834065 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.834545 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.835284 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.835600 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.835705 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.838011 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:45.838014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:45.838017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:45.838045 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:45.839523 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.839983 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.839988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.839989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.840060 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:45.840313 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:45.840316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:45.840318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:45.840468 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:45.840568 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:45.840826 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 18:17:45.840974 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":53}
I20260808 18:17:45.841092 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:45.832764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:37267 (local address 127.0.0.1:33384)
0808 18:17:45.833090 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:45.833102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:45.833222 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:45.833465 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:45.833468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:45.833857 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:45.833951 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.833957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.834644 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.834649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.835164 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:45.835172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:45.835707 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:45.835710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:45.835787 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:45.836452 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:45.836464 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:45.837886 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:45.839639 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.839649 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.839655 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.839891 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.840144 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:45.840148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:45.840150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:45.840225 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:45.840568 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:45.840575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:45.840674 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 18:17:45.840782 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 18:17:45.840924 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":125,"threads_started":1}
I20260808 18:17:45.844991 29589 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33384:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:17:45.845652 29589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:45.850188 30058 tablet_service.cc:1467] Processing CreateTablet for tablet bfb6f93529bb4ea3abf465c055f01982 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e1f50e468cf43da8f39722885bdf382]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:17:45.850404 30276 tablet_service.cc:1467] Processing CreateTablet for tablet bfb6f93529bb4ea3abf465c055f01982 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e1f50e468cf43da8f39722885bdf382]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:17:45.850675 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb6f93529bb4ea3abf465c055f01982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.850800 30276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb6f93529bb4ea3abf465c055f01982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.850188 29830 tablet_service.cc:1467] Processing CreateTablet for tablet bfb6f93529bb4ea3abf465c055f01982 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e1f50e468cf43da8f39722885bdf382]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:17:45.852933 30414 tablet_bootstrap.cc:492] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: Bootstrap starting.
I20260808 18:17:45.853129 29830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb6f93529bb4ea3abf465c055f01982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:45.853788 30414 tablet_bootstrap.cc:654] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.854442 30417 tablet_bootstrap.cc:492] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: Bootstrap starting.
I20260808 18:17:45.854952 30414 tablet_bootstrap.cc:492] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: No bootstrap required, opened a new log
I20260808 18:17:45.855052 30414 ts_tablet_manager.cc:1397] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:45.855584 30414 raft_consensus.cc:348] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.855705 30414 raft_consensus.cc:374] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.855752 30414 raft_consensus.cc:729] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9f6a0091684361b7bfe733383819e7, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.855887 30414 consensus_queue.cc:260] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.856197 30414 ts_tablet_manager.cc:1428] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:17:45.856274 30417 tablet_bootstrap.cc:654] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.856228 30449 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:45.856343 30418 tablet_bootstrap.cc:492] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: Bootstrap starting.
I20260808 18:17:45.856371 30449 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:45.857324 30418 tablet_bootstrap.cc:654] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:45.858570 30418 tablet_bootstrap.cc:492] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: No bootstrap required, opened a new log
I20260808 18:17:45.858657 30418 ts_tablet_manager.cc:1397] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:45.858999 30418 raft_consensus.cc:348] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.859124 30418 raft_consensus.cc:374] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.859149 30418 raft_consensus.cc:729] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461033e6b2174cd48c156ea67c33523d, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.859261 30418 consensus_queue.cc:260] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.859537 30418 ts_tablet_manager.cc:1428] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:17:45.859562 30428 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:45.859714 30428 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:45.860152 30417 tablet_bootstrap.cc:492] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: No bootstrap required, opened a new log
I20260808 18:17:45.860247 30417 ts_tablet_manager.cc:1397] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 18:17:45.860620 30417 raft_consensus.cc:348] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.860740 30417 raft_consensus.cc:374] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:45.860778 30417 raft_consensus.cc:729] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626c770d2a5d4503a28d1b3cf4835cc6, State: Initialized, Role: FOLLOWER
I20260808 18:17:45.860937 30417 consensus_queue.cc:260] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:45.861349 30417 ts_tablet_manager.cc:1428] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:17:45.861311 30431 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:45.861447 30431 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:46.146143 30428 raft_consensus.cc:482] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:46.146279 30428 raft_consensus.cc:504] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:46.147433 30428 leader_election.cc:288] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595), 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565)
I20260808 18:17:46.152462 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.147721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:37643 (local address 127.27.18.194:44595)
0808 18:17:46.147859 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.147863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.147885 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:17:46.148109 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.148115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.148172 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.148260 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:46.148665 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.149476 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.151197 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.151781 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.151899 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.151951 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.152264 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.152343 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:17:46.152464 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.147856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:59513 (local address 127.27.18.195:42565)
0808 18:17:46.148100 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.148105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.148118 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:17:46.148166 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.148168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.148217 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.148301 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:46.148655 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.149466 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.151197 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.151788 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.151899 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.151951 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.152264 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.152339 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260808 18:17:46.152495 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.147650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42565 (local address 127.27.18.193:59513)
0808 18:17:46.147955 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:46.147969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:46.148043 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:46.148320 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:46.148322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:46.148345 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:46.148535 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.148540 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.149588 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.149591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.151071 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.151084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.151908 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.151912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.152106 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.152157 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 18:17:46.152233 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:17:46.152332 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":303,"spinlock_wait_cycles":1152,"thread_start_us":70,"threads_started":1}
I20260808 18:17:46.152504 30487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.147593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:44595 (local address 127.27.18.193:37643)
0808 18:17:46.147825 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:46.147839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:46.147938 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:46.148298 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:46.148301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:46.148330 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:46.148511 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.148535 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.149588 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.149591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.151077 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.151084 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.151908 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.151911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.152100 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.152156 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 18:17:46.152233 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:17:46.152332 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":253,"spinlock_wait_cycles":5888,"thread_start_us":73,"threads_started":1}
I20260808 18:17:46.153017 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb6f93529bb4ea3abf465c055f01982" candidate_uuid: "461033e6b2174cd48c156ea67c33523d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9f6a0091684361b7bfe733383819e7" is_pre_election: true
I20260808 18:17:46.153141 30091 raft_consensus.cc:2454] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461033e6b2174cd48c156ea67c33523d in term 0.
I20260808 18:17:46.153275 30304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb6f93529bb4ea3abf465c055f01982" candidate_uuid: "461033e6b2174cd48c156ea67c33523d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" is_pre_election: true
I20260808 18:17:46.153427 30304 raft_consensus.cc:2454] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461033e6b2174cd48c156ea67c33523d in term 0.
I20260808 18:17:46.153541 29747 leader_election.cc:302] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461033e6b2174cd48c156ea67c33523d, 8c9f6a0091684361b7bfe733383819e7; no voters:
I20260808 18:17:46.153805 30428 raft_consensus.cc:2781] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:46.153862 30428 raft_consensus.cc:482] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:46.153889 30428 raft_consensus.cc:3037] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:46.154834 30428 raft_consensus.cc:504] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:46.155208 30428 leader_election.cc:288] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [CANDIDATE]: Term 1 election: Requested vote from peers 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595), 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565)
I20260808 18:17:46.155520 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb6f93529bb4ea3abf465c055f01982" candidate_uuid: "461033e6b2174cd48c156ea67c33523d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9f6a0091684361b7bfe733383819e7"
I20260808 18:17:46.155633 30304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb6f93529bb4ea3abf465c055f01982" candidate_uuid: "461033e6b2174cd48c156ea67c33523d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "626c770d2a5d4503a28d1b3cf4835cc6"
I20260808 18:17:46.155666 30091 raft_consensus.cc:3037] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:46.155748 30304 raft_consensus.cc:3037] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:46.156548 30091 raft_consensus.cc:2454] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461033e6b2174cd48c156ea67c33523d in term 1.
I20260808 18:17:46.156591 30304 raft_consensus.cc:2454] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461033e6b2174cd48c156ea67c33523d in term 1.
I20260808 18:17:46.156800 29747 leader_election.cc:302] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461033e6b2174cd48c156ea67c33523d, 8c9f6a0091684361b7bfe733383819e7; no voters:
I20260808 18:17:46.157034 30428 raft_consensus.cc:2781] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:46.157176 30428 raft_consensus.cc:686] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [term 1 LEADER]: Becoming Leader. State: Replica: 461033e6b2174cd48c156ea67c33523d, State: Running, Role: LEADER
I20260808 18:17:46.157442 30428 consensus_queue.cc:237] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } }
I20260808 18:17:46.158599 30493 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: TxnStatusTablet state changed. Reason: New leader 461033e6b2174cd48c156ea67c33523d. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.158754 30493 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:17:46.159113 30496 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:17:46.159332 30496 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:17:46.159789 29589 catalog_manager.cc:5495] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d reported cstate change: term changed from 0 to 1, leader changed from <none> to 461033e6b2174cd48c156ea67c33523d (127.27.18.193). New cstate: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } health_report { overall_health: UNKNOWN } } }
I20260808 18:17:46.217861 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.213597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:36167 (local address 127.0.0.1:39138)
0808 18:17:46.213753 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:46.213771 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:46.213881 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:46.214239 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:46.214242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:46.214250 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:46.214356 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.214362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.215194 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.215198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.216046 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.216055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.216612 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.216615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.216701 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.216752 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.217490 (+ 738us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.217510 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:17:46.217621 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 18:17:46.217706 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 18:17:46.218230 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.213728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.27.18.193:36167)
0808 18:17:46.213859 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.213864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.213878 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:17:46.214043 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.214046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.214108 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.214211 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:46.214486 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.215083 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.216155 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.216483 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.216613 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.216934 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.217381 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.217476 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.218041 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.218111 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:17:46.221251 30304 raft_consensus.cc:1261] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Refusing update from remote peer 461033e6b2174cd48c156ea67c33523d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:46.221504 30091 raft_consensus.cc:1261] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Refusing update from remote peer 461033e6b2174cd48c156ea67c33523d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:46.221751 30428 consensus_queue.cc:1035] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [LEADER]: Connected to new peer: Peer: permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:46.222069 30493 consensus_queue.cc:1035] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:46.223456 30431 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: TxnStatusTablet state changed. Reason: New leader 461033e6b2174cd48c156ea67c33523d. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.223614 30431 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:46.225306 30449 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: TxnStatusTablet state changed. Reason: New leader 461033e6b2174cd48c156ea67c33523d. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.225419 30449 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:46.228098 30507 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.228305 30507 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:17:46.228286 30503 mvcc.cc:205] Tried to move back new op lower bound from 7316328719237353472 to 7316328718982995968. Current Snapshot: MvccSnapshot[applied={T|T < 7316328719237353472}]
I20260808 18:17:46.228448 30449 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.228561 30449 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 8c9f6a0091684361b7bfe733383819e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:46.228642 30507 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.229043 30507 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 461033e6b2174cd48c156ea67c33523d: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:17:46.229030 30431 tablet_replica.cc:439] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "461033e6b2174cd48c156ea67c33523d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 } } peers { permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 } } peers { permanent_uuid: "626c770d2a5d4503a28d1b3cf4835cc6" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42565 } } }
I20260808 18:17:46.229140 30431 tablet_replica.cc:442] T bfb6f93529bb4ea3abf465c055f01982 P 626c770d2a5d4503a28d1b3cf4835cc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:46.238967 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.235497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.27.18.195:42565)
0808 18:17:46.235601 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.235605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.235832 (+ 227us) server_negotiation.cc:407] Connection header received
0808 18:17:46.235981 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.235984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.236043 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.236155 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:46.236407 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.236852 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.237646 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.237854 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.237969 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.238256 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.238594 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.238716 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.238800 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.238851 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:17:46.249552 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.246093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:56069 (local address 127.27.18.193:36167)
0808 18:17:46.246216 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.246239 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.246471 (+ 232us) server_negotiation.cc:407] Connection header received
0808 18:17:46.246660 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.246663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.246703 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.246791 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:46.247065 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.247586 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.248412 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.248655 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.248779 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.249112 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.249185 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.249276 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.249378 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.249424 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 18:17:46.249605 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.245997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:36167 (local address 127.27.18.195:56069)
0808 18:17:46.246371 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:46.246382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:46.246516 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:46.246803 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:46.246806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:46.246814 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:46.246945 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.246951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.247712 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.247715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.248277 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.248286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.248795 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.248799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.248904 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.248955 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.249281 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.249292 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 18:17:46.249368 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:17:46.249461 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":86,"threads_started":1}
I20260808 18:17:46.257201 30091 raft_consensus.cc:1261] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Refusing update from remote peer 626c770d2a5d4503a28d1b3cf4835cc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:46.257236 29869 raft_consensus.cc:1261] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Refusing update from remote peer 626c770d2a5d4503a28d1b3cf4835cc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:46.257766 30471 consensus_queue.cc:1035] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "461033e6b2174cd48c156ea67c33523d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 36167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:46.257944 30431 consensus_queue.cc:1035] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9f6a0091684361b7bfe733383819e7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:46.290169 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.285830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:53137 (local address 127.27.18.195:42565)
0808 18:17:46.285927 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.285931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.285944 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:17:46.286048 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.286050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.286098 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.286174 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:46.286522 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.287192 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.288538 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.288916 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.289002 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.289618 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.289703 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.289775 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.289977 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.290035 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:17:46.290840 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.285481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42565 (local address 127.27.18.193:53137)
0808 18:17:46.285849 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:46.285864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:46.285962 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:46.286212 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:46.286216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:46.286227 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:46.286391 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.286401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.287321 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.287326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.288366 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:46.288378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.289279 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.289282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:46.289392 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.289447 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:46.289850 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:46.289855 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:17:46.290562 (+ 707us) client_negotiation.cc:240] Negotiation successful
0808 18:17:46.290647 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
I20260808 18:17:46.322841 29587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:46.323043 29587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:46.325665 29587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a0a524e3e6647beb31be7838bfb1c56: Sending DeleteTablet for 3 replicas of tablet 445b6c4929c04dbd8b1244e3643d7980
I20260808 18:17:46.326319 30276 tablet_service.cc:1514] Processing DeleteTablet for tablet 445b6c4929c04dbd8b1244e3643d7980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:46 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50788
I20260808 18:17:46.326412 30058 tablet_service.cc:1514] Processing DeleteTablet for tablet 445b6c4929c04dbd8b1244e3643d7980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:46 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:43556
I20260808 18:17:46.326566 29830 tablet_service.cc:1514] Processing DeleteTablet for tablet 445b6c4929c04dbd8b1244e3643d7980 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:46 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:39124
I20260808 18:17:46.326781 30539 tablet_replica.cc:330] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: stopping tablet replica
I20260808 18:17:46.326891 30541 tablet_replica.cc:330] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: stopping tablet replica
I20260808 18:17:46.326999 30541 raft_consensus.cc:2229] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:46.327105 30541 raft_consensus.cc:2258] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:46.327498 30539 raft_consensus.cc:2229] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:17:46.327762 30541 ts_tablet_manager.cc:1905] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:46.327804 30539 raft_consensus.cc:2258] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:46.328467 30539 ts_tablet_manager.cc:1905] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:46.329981 30542 tablet_replica.cc:330] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: stopping tablet replica
I20260808 18:17:46.330154 30542 raft_consensus.cc:2229] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:46.330297 30542 raft_consensus.cc:2258] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:46.331118 30542 ts_tablet_manager.cc:1905] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:46.331691 30539 ts_tablet_manager.cc:1918] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:46.331812 30539 log.cc:1196] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/445b6c4929c04dbd8b1244e3643d7980
I20260808 18:17:46.332134 30539 ts_tablet_manager.cc:1939] T 445b6c4929c04dbd8b1244e3643d7980 P 626c770d2a5d4503a28d1b3cf4835cc6: Deleting consensus metadata
I20260808 18:17:46.332849 29566 catalog_manager.cc:4841] TS 626c770d2a5d4503a28d1b3cf4835cc6 (127.27.18.195:42565): tablet 445b6c4929c04dbd8b1244e3643d7980 (table test-table [id=97a0079872ed4d07b6721adf8a34848f]) successfully deleted
I20260808 18:17:46.334295 30542 ts_tablet_manager.cc:1918] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:46.334384 30542 log.cc:1196] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/445b6c4929c04dbd8b1244e3643d7980
I20260808 18:17:46.334689 30542 ts_tablet_manager.cc:1939] T 445b6c4929c04dbd8b1244e3643d7980 P 461033e6b2174cd48c156ea67c33523d: Deleting consensus metadata
I20260808 18:17:46.335434 29569 catalog_manager.cc:4841] TS 461033e6b2174cd48c156ea67c33523d (127.27.18.193:36167): tablet 445b6c4929c04dbd8b1244e3643d7980 (table test-table [id=97a0079872ed4d07b6721adf8a34848f]) successfully deleted
I20260808 18:17:46.336661 30541 ts_tablet_manager.cc:1918] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:46.336747 30541 log.cc:1196] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/445b6c4929c04dbd8b1244e3643d7980
I20260808 18:17:46.337021 30541 ts_tablet_manager.cc:1939] T 445b6c4929c04dbd8b1244e3643d7980 P 8c9f6a0091684361b7bfe733383819e7: Deleting consensus metadata
I20260808 18:17:46.337687 29567 catalog_manager.cc:4841] TS 8c9f6a0091684361b7bfe733383819e7 (127.27.18.194:44595): tablet 445b6c4929c04dbd8b1244e3643d7980 (table test-table [id=97a0079872ed4d07b6721adf8a34848f]) successfully deleted
Aug 08 18:17:46 dist-test-slave-x4qh krb5kdc[29516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213065, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:17:46.345683 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.330910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.27.18.193:36167)
0808 18:17:46.331034 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.331039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.335329 (+ 4290us) server_negotiation.cc:407] Connection header received
0808 18:17:46.335614 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.335616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.335681 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.335762 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:46.336704 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.337485 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.338391 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.338601 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.338711 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.341985 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:46.341996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:46.342009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:46.342040 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:46.343909 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:46.344385 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:46.344390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:46.344392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:46.344472 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:46.344761 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:46.344765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:46.344767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:46.345014 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:46.345238 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.345404 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.345522 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:17:46.346167 29932 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39146
I20260808 18:17:46.355051 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:46.347821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.27.18.254:37267)
0808 18:17:46.348060 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:46.348064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:46.348104 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:17:46.348140 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:46.348142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:46.348186 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:46.348278 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:46.348859 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.349655 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.350322 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:46.350644 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:46.350733 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:46.351914 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:46.351917 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:46.351920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:46.351943 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:46.353513 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:46.353918 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:46.353922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:46.353924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:46.354023 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:46.354261 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:46.354264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:46.354266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:46.354452 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:46.354576 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:46.354779 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 18:17:46.354906 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":65,"threads_started":1}
W20260808 18:17:46.355463 29587 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33392
I20260808 18:17:46.356624 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29712
I20260808 18:17:46.395809 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29952
I20260808 18:17:46.417171 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30176
I20260808 18:17:46.425559 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29538
2026-08-08T18:17:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3226 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:46 dist-test-slave-x4qh krb5kdc[30557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:46 dist-test-slave-x4qh krb5kdc[30557](info): set up 2 sockets
Aug 08 18:17:46 dist-test-slave-x4qh krb5kdc[30557](info): commencing operation
krb5kdc: starting...
W20260808 18:17:47.618249 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:47 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:47.734084 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41389
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:44345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:47.841059 30580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:47.841320 30580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:47.841396 30580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:47.845234 30580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:47.845312 30580 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:17:47.845345 30580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:47.845404 30580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:47.845443 30580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:47.845480 30580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:47.850422 30580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:41389
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:47.851352 30580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:47.852329 30580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:47.860491 30587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:47.860565 30580 server_base.cc:1034] running on GCE node
W20260808 18:17:47.860502 30588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:47.860502 30592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:47.861337 30580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:47.862262 30580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:47.863449 30580 hybrid_clock.cc:648] HybridClock initialized: now 1786213067863437 us; error 32 us; skew 500 ppm
Aug 08 18:17:47 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213067, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:47.869953 30580 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:47.870828 30580 webserver.cc:460] Webserver started at http://127.27.18.254:36209/ using document root <none> and password file <none>
I20260808 18:17:47.871435 30580 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:47.871527 30580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:47.871759 30580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:47.873668 30580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "4e2d4a9aa81c461495863fb329aefdc4"
format_stamp: "Formatted at 2026-08-08 18:17:47 on dist-test-slave-x4qh"
server_key: "0e5396e22151baece3409f5df3b8a8a4"
server_key_iv: "b7d0dcbddfff4d455b18351415ee17eb"
server_key_version: "encryptionkey@0"
I20260808 18:17:47.874217 30580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "4e2d4a9aa81c461495863fb329aefdc4"
format_stamp: "Formatted at 2026-08-08 18:17:47 on dist-test-slave-x4qh"
server_key: "0e5396e22151baece3409f5df3b8a8a4"
server_key_iv: "b7d0dcbddfff4d455b18351415ee17eb"
server_key_version: "encryptionkey@0"
I20260808 18:17:47.877671 30580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:17:47.879755 30607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:47.880769 30580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:47.880910 30580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4e2d4a9aa81c461495863fb329aefdc4"
format_stamp: "Formatted at 2026-08-08 18:17:47 on dist-test-slave-x4qh"
server_key: "0e5396e22151baece3409f5df3b8a8a4"
server_key_iv: "b7d0dcbddfff4d455b18351415ee17eb"
server_key_version: "encryptionkey@0"
I20260808 18:17:47.881031 30580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:47.896420 30580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:47.898881 30580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:47.899065 30580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:47.906440 30580 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:41389
I20260808 18:17:47.906463 30712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:41389 every 8 connection(s)
I20260808 18:17:47.907259 30580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:47.909317 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30580
I20260808 18:17:47.909427 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:47.909699 27723 external_mini_cluster.cc:1428] Setting key 2479bcc80b7b90c6c96ab577d992828e
I20260808 18:17:47.910137 30714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:47.915781 30714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Bootstrap starting.
Aug 08 18:17:47 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:47.917760 30714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:47.918567 30714 log.cc:824] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:47.920897 30714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: No bootstrap required, opened a new log
I20260808 18:17:47.923743 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:47.911026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36052 (local address 127.27.18.254:41389)
0808 18:17:47.911494 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:47.911512 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:47.911555 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:17:47.912116 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:47.912129 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:47.912406 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:47.912721 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:47.913524 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:47.914459 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:47.915269 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:47.915789 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:47.915978 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:47.918308 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:47.918314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:47.918328 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:47.918365 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:47.920921 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:47.921338 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:47.921344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:47.921349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:47.921484 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:47.921730 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:47.921733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:47.921734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:47.922169 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:47.922330 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:47.922557 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 18:17:47.922862 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":129,"threads_started":1}
I20260808 18:17:47.924575 30714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } }
I20260808 18:17:47.924821 30714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:47.924896 30714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2d4a9aa81c461495863fb329aefdc4, State: Initialized, Role: FOLLOWER
I20260808 18:17:47.925309 30714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } }
I20260808 18:17:47.925469 30719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:47.925618 30719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } }
I20260808 18:17:47.925971 30714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:47.925959 30719 leader_election.cc:302] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e2d4a9aa81c461495863fb329aefdc4; no voters:
I20260808 18:17:47.926080 30714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:47.926129 30714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:47.926383 30719 leader_election.cc:288] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260808 18:17:47.926954 30714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } }
I20260808 18:17:47.927094 30714 leader_election.cc:302] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e2d4a9aa81c461495863fb329aefdc4; no voters:
I20260808 18:17:47.927139 30722 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 18:17:47.927201 30714 leader_election.cc:288] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:47.927474 30722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } } }
I20260808 18:17:47.927631 30722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:17:47.927724 30719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:47.927830 30719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e2d4a9aa81c461495863fb329aefdc4, State: Running, Role: LEADER
I20260808 18:17:47.928054 30719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } }
I20260808 18:17:47.928254 30714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:47.929000 30719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e2d4a9aa81c461495863fb329aefdc4. Latest consensus state: current_term: 1 leader_uuid: "4e2d4a9aa81c461495863fb329aefdc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2d4a9aa81c461495863fb329aefdc4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41389 } } }
I20260808 18:17:47.929088 30719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4 [sys.catalog]: This master's current role is: LEADER
W20260808 18:17:47.932931 30741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:17:47.933019 30741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:17:47.933101 30733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:47.934016 30733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:47.939438 30733 catalog_manager.cc:1347] Generated new cluster ID: a60c7fd56bcc44678bed77c7628e7cc2
I20260808 18:17:47.939498 30733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:47.960702 30733 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:47.961804 30733 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:47.975059 30733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Generated new TSK 0
I20260808 18:17:47.975728 30733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:17:48.010717 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41389
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:44345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:48.116389 30758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:48.116597 30758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:48.116664 30758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:48.120364 30758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:48.120409 30758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:48.120465 30758 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:17:48.120496 30758 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:17:48.120565 30758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:17:48.125499 30758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:41389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:48.126385 30758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:48.127427 30758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:48.136777 30766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:48.136786 30767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:48.136883 30758 server_base.cc:1034] running on GCE node
W20260808 18:17:48.136883 30770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:48.137526 30758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:48.138129 30758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:48.139338 30758 hybrid_clock.cc:648] HybridClock initialized: now 1786213068139318 us; error 40 us; skew 500 ppm
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:48.142031 30758 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:17:48.142724 30758 webserver.cc:460] Webserver started at http://127.27.18.193:41793/ using document root <none> and password file <none>
I20260808 18:17:48.143195 30758 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:48.143250 30758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:48.143467 30758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:48.145165 30758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "f9a4e8560bef4465a506278442ac1851"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "4f0ea5aa34e5dfa4943df74296ea2440"
server_key_iv: "c249316bdb445b8b5075ae0544716afe"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.145687 30758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "f9a4e8560bef4465a506278442ac1851"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "4f0ea5aa34e5dfa4943df74296ea2440"
server_key_iv: "c249316bdb445b8b5075ae0544716afe"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.148846 30758 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:17:48.151153 30782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:48.152307 30758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:17:48.152405 30758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "f9a4e8560bef4465a506278442ac1851"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "4f0ea5aa34e5dfa4943df74296ea2440"
server_key_iv: "c249316bdb445b8b5075ae0544716afe"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.152482 30758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:48.165510 30758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:48.168450 30758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:48.168581 30758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:48.169795 30758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:48.169854 30758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:48.169891 30758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:48.169960 30758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:48.179929 30758 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:38681
I20260808 18:17:48.180109 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:38681 every 8 connection(s)
I20260808 18:17:48.181018 30758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:17:48.183827 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.170586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:47879 (local address 127.27.18.254:41389)
0808 18:17:48.170720 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.170725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.171530 (+ 805us) server_negotiation.cc:407] Connection header received
0808 18:17:48.172395 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.172400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.172469 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.172555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.174080 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.174957 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.176861 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.177277 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.177396 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.180052 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.180056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.180059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.180083 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.181652 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.182215 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.182218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.182220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.182353 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.182732 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.182735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.182737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.182927 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.183038 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.183559 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.183687 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:17:48.184545 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.170917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.27.18.193:47879)
0808 18:17:48.171349 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.171422 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.172188 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.172814 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.172825 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.173236 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.173879 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.173903 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.175107 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.175112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.176723 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.176735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.177436 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.177440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.177593 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.178063 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.178077 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.179874 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.181762 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.181767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.181784 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.182088 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.182456 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.182459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.182461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.182623 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.183037 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.183044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.183266 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.183474 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.183791 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":124,"threads_started":1}
I20260808 18:17:48.185814 30964 heartbeater.cc:347] Connected to a master server at 127.27.18.254:41389
I20260808 18:17:48.185976 30964 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:48.186234 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30758
I20260808 18:17:48.186353 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:17:48.186502 30964 heartbeater.cc:511] Master 127.27.18.254:41389 requested a full tablet report, sending...
I20260808 18:17:48.186722 27723 external_mini_cluster.cc:1428] Setting key 65248f801ecff58ebe17dd68bcc00e6a
I20260808 18:17:48.188220 30640 ts_manager.cc:194] Registered new tserver with Master: f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681)
I20260808 18:17:48.189571 30640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:47879
I20260808 18:17:48.195236 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.188176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:44399 (local address 127.27.18.254:41389)
0808 18:17:48.188279 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.188283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.188514 (+ 231us) server_negotiation.cc:407] Connection header received
0808 18:17:48.188621 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.188623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.188664 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.188726 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.189205 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.189672 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.190743 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.191059 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.191156 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.192394 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.192398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.192400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.192425 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.193719 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.194132 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.194144 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.194146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.194224 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.194503 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.194507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.194509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.194661 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.194760 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.194998 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.195099 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:17:48.195248 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.188153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.27.18.193:44399)
0808 18:17:48.188441 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.188454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.188546 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.188770 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.188773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.188990 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.189098 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.189104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.189832 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.189836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.190614 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.190629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.191261 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.191264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.191346 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.191849 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.191864 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.192258 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.193833 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.193837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.193839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.194028 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.194317 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.194320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.194322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.194416 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.194753 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.194756 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.194848 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.194948 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.195070 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:17:48.215801 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41389
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:44345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:48.326344 30975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:48.326574 30975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:48.326619 30975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:48.330729 30975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:48.330781 30975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:48.330873 30975 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:17:48.330917 30975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:17:48.330984 30975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:17:48.336373 30975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:41389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:48.337296 30975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:48.338243 30975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:48.347476 30987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:48.347450 30981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:48.347460 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:48.347709 30975 server_base.cc:1034] running on GCE node
I20260808 18:17:48.348235 30975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:48.348769 30975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:48.349977 30975 hybrid_clock.cc:648] HybridClock initialized: now 1786213068349957 us; error 29 us; skew 500 ppm
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:48.352629 30975 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:17:48.353489 30975 webserver.cc:460] Webserver started at http://127.27.18.194:39675/ using document root <none> and password file <none>
I20260808 18:17:48.354013 30975 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:48.354074 30975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:48.354314 30975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:48.356330 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "dfea4f0710364c53b3bd70fea8b06e22"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "756b13c46f5bc8f82f18853f44f16070"
server_key_iv: "fa728f6f3a2caf64a809c05bc4306a9d"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.356894 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "dfea4f0710364c53b3bd70fea8b06e22"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "756b13c46f5bc8f82f18853f44f16070"
server_key_iv: "fa728f6f3a2caf64a809c05bc4306a9d"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.360306 30975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:17:48.362578 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:48.363799 30975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:17:48.363922 30975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "dfea4f0710364c53b3bd70fea8b06e22"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "756b13c46f5bc8f82f18853f44f16070"
server_key_iv: "fa728f6f3a2caf64a809c05bc4306a9d"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.364025 30975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:48.377122 30975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:48.379657 30975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:48.379873 30975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:48.381316 30975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:48.381363 30975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:48.381417 30975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:48.381488 30975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:48.393186 30975 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33031
I20260808 18:17:48.393214 31188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33031 every 8 connection(s)
I20260808 18:17:48.393965 30975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:17:48.395588 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.382006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:34377 (local address 127.27.18.254:41389)
0808 18:17:48.382178 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.382183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.383000 (+ 817us) server_negotiation.cc:407] Connection header received
0808 18:17:48.383945 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.383948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.384007 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.384088 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.385915 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.386656 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.388053 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.388287 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.388406 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.391884 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.391888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.391891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.391928 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.393573 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.394064 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.394068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.394070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.394152 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.394524 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.394527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.394529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.394738 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.394837 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.395320 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.395475 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 18:17:48.396440 31016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.382397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.27.18.194:34377)
0808 18:17:48.382828 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.382875 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.383731 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.384411 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.384420 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.384947 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.385709 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.385733 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.386845 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.386850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.387926 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.387936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.388455 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.388459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.388671 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.389224 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.389245 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.391727 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.393690 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.393695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.393707 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.393954 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.394244 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.394248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.394250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.394384 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.394855 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.394861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.395103 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.395251 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.395549 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":124,"threads_started":1}
I20260808 18:17:48.397794 31189 heartbeater.cc:347] Connected to a master server at 127.27.18.254:41389
I20260808 18:17:48.397950 31189 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:48.398419 31189 heartbeater.cc:511] Master 127.27.18.254:41389 requested a full tablet report, sending...
I20260808 18:17:48.399437 30634 ts_manager.cc:194] Registered new tserver with Master: dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:48.400166 30634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:34377
I20260808 18:17:48.401412 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30975
I20260808 18:17:48.401535 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:17:48.401832 27723 external_mini_cluster.cc:1428] Setting key 5f4139ee4571e2d20532af156edb4a5a
I20260808 18:17:48.407222 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.399915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:58799 (local address 127.27.18.254:41389)
0808 18:17:48.400074 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.400078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.400226 (+ 148us) server_negotiation.cc:407] Connection header received
0808 18:17:48.400335 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.400338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.400402 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.400488 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.400963 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.401508 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.402483 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.402687 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.402786 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.403935 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.403938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.403939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.403960 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.405607 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.405979 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.405983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.405988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.406073 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.406349 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.406355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.406357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.406646 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.406750 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.406984 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.407090 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:17:48.407222 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.399854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.27.18.194:58799)
0808 18:17:48.400150 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.400165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.400263 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.400505 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.400508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.400687 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.400816 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.400823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.401684 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.401687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.402376 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.402386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.402800 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.402803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.402898 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.403336 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.403350 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.403794 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.405717 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.405719 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.405720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.405887 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.406164 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.406166 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.406167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.406253 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.406742 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.406744 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.406814 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.406924 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.407054 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:17:48.428450 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41389
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:44345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:48.535966 31201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:48.536226 31201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:48.536303 31201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:48.540246 31201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:48.540316 31201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:48.540382 31201 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 18:17:48.540426 31201 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:17:48.540490 31201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:17:48.545565 31201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:41389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:48.546501 31201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:48.547480 31201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:48.556762 31211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:48.556763 31207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:48.556798 31206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:48.557021 31201 server_base.cc:1034] running on GCE node
I20260808 18:17:48.557417 31201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:48.557960 31201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:48.559118 31201 hybrid_clock.cc:648] HybridClock initialized: now 1786213068559099 us; error 29 us; skew 500 ppm
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:48.561723 31201 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:17:48.562526 31201 webserver.cc:460] Webserver started at http://127.27.18.195:38165/ using document root <none> and password file <none>
I20260808 18:17:48.563043 31201 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:48.563102 31201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:48.563316 31201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:48.565277 31201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "2246cfd85da04e38911906e3ac13c208"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "f830c1b14e31d4db4afbf5a5cca2ace6"
server_key_iv: "fd618dc140f88c1b912293bbcc8ef63e"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.565846 31201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "2246cfd85da04e38911906e3ac13c208"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "f830c1b14e31d4db4afbf5a5cca2ace6"
server_key_iv: "fd618dc140f88c1b912293bbcc8ef63e"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.569635 31201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:17:48.571979 31222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:48.573103 31201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:17:48.573225 31201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "2246cfd85da04e38911906e3ac13c208"
format_stamp: "Formatted at 2026-08-08 18:17:48 on dist-test-slave-x4qh"
server_key: "f830c1b14e31d4db4afbf5a5cca2ace6"
server_key_iv: "fd618dc140f88c1b912293bbcc8ef63e"
server_key_version: "encryptionkey@0"
I20260808 18:17:48.573325 31201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:48.582906 31201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:48.585996 31201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:48.586222 31201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:48.587662 31201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:48.587734 31201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:48.587803 31201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:48.587849 31201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:48.598901 31201 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:38499
I20260808 18:17:48.598984 31424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:38499 every 8 connection(s)
I20260808 18:17:48.599978 31201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:17:48.601471 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.588343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:54305 (local address 127.27.18.254:41389)
0808 18:17:48.588539 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.588545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.589795 (+ 1250us) server_negotiation.cc:407] Connection header received
0808 18:17:48.590818 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.590824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.590890 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.590994 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.592189 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.593047 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.594482 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.594784 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.594896 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.597663 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.597667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.597669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.597693 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.599351 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.599953 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.599956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.599958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.600063 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.600400 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.600404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.600406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.600639 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.600736 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.601270 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.601363 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 18:17:48.602154 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.589213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.27.18.195:54305)
0808 18:17:48.589637 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.589685 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.590638 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.591224 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.591238 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.591621 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.592027 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.592040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.593175 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.593180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.594356 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.594366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.594942 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.594947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.595122 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.595660 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.595690 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.597496 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.599507 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.599512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.599524 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.599849 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.600170 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.600173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.600175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.600300 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.600795 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.600801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.601050 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.601201 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.601410 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
I20260808 18:17:48.603291 31429 heartbeater.cc:347] Connected to a master server at 127.27.18.254:41389
I20260808 18:17:48.603475 31429 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:48.604051 31429 heartbeater.cc:511] Master 127.27.18.254:41389 requested a full tablet report, sending...
I20260808 18:17:48.604523 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 31201
I20260808 18:17:48.604663 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:17:48.604977 27723 external_mini_cluster.cc:1428] Setting key d21aeb9b641bfef160d1df8fe68886cc
I20260808 18:17:48.605259 30640 ts_manager.cc:194] Registered new tserver with Master: 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499)
I20260808 18:17:48.606037 30640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:54305
I20260808 18:17:48.607143 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:17:48.613176 31435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.605922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.27.18.195:42141)
0808 18:17:48.606315 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.606329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.606451 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.606743 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.606745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.606953 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.607046 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.607052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.607902 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.607905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.608435 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.608443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.608876 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.608880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.608965 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.609446 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.609466 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.609913 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.611782 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.611786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.611788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.611983 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.612264 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.612268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.612270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.612358 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.612796 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.612799 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.612885 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.612969 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.613063 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":104,"threads_started":1}
I20260808 18:17:48.613416 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.605987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:42141 (local address 127.27.18.254:41389)
0808 18:17:48.606111 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.606116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.606391 (+ 275us) server_negotiation.cc:407] Connection header received
0808 18:17:48.606559 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.606562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.606616 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.606715 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.607163 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.607777 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.608552 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.608768 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.608872 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.610070 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.610073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.610075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.610105 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.611674 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.612085 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.612090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.612091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.612170 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.612451 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.612455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.612457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.612619 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.612746 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.613078 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.613214 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:48.635208 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.627613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.27.18.254:41389)
0808 18:17:48.627745 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.627748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.627767 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:17:48.627795 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.627797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.627829 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.627912 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.628604 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.629245 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.630175 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.630426 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.630515 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.632670 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.632673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.632674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.632695 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.633869 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.634241 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.634243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.634244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.634309 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.634524 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.634526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.634527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.634663 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.634759 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.634937 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.635021 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:17:48.637951 30640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:17:48.640882 30640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:48.654312 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.648547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:38499 (local address 127.0.0.1:52192)
0808 18:17:48.648939 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.648982 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.649092 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.649855 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.649862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.649894 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.650036 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.650045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.651633 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.651635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.652727 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.652739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.653784 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.653788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.653904 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.653947 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.654058 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.654192 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":63,"threads_started":1}
I20260808 18:17:48.654325 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.648646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.27.18.195:38499)
0808 18:17:48.649344 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.649358 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.649386 (+ 28us) server_negotiation.cc:407] Connection header received
0808 18:17:48.649473 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.649479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.649681 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.649848 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.650206 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.651500 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.652876 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.653679 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.653787 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.653952 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.654091 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.654191 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":97,"threads_started":1}
I20260808 18:17:48.655001 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.648664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33031 (local address 127.0.0.1:37930)
0808 18:17:48.649310 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.649321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.649423 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.650620 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.650623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.650642 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.650754 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.650759 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.652317 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.652320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.653597 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.653609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.654556 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.654560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.654668 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.654715 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.654811 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.654884 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":67,"thread_start_us":75,"threads_started":1}
I20260808 18:17:48.655095 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.649204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.27.18.194:33031)
0808 18:17:48.650152 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.650164 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.650187 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:17:48.650271 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.650276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.650468 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.650880 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.650985 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.652185 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.653710 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.654449 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.654556 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.654717 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.654845 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.654934 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":120,"threads_started":1}
I20260808 18:17:48.655232 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.649306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38681 (local address 127.0.0.1:34334)
0808 18:17:48.649911 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.649925 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.650019 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.650840 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.650843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.650860 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.650964 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.650970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.652374 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.652377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.653530 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.653539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.654828 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.654832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.654926 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.654947 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.655017 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.655076 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":92,"thread_start_us":104,"threads_started":1}
I20260808 18:17:48.655732 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.649752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.27.18.193:38681)
0808 18:17:48.650206 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.650219 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.650246 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:17:48.650324 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.650329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.650512 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.650691 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.651099 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.652229 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.653651 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.654442 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.654556 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.654717 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.655470 (+ 753us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.655597 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":107,"threads_started":1}
I20260808 18:17:48.657722 31080 tablet_service.cc:1467] Processing CreateTablet for tablet 27417c03d11b400b9ec756ca941432ed (DEFAULT_TABLE table=test-table [id=209cd7a3c3344730a035c0590e853779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:48.657670 31292 tablet_service.cc:1467] Processing CreateTablet for tablet 27417c03d11b400b9ec756ca941432ed (DEFAULT_TABLE table=test-table [id=209cd7a3c3344730a035c0590e853779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:48.658149 30859 tablet_service.cc:1467] Processing CreateTablet for tablet 27417c03d11b400b9ec756ca941432ed (DEFAULT_TABLE table=test-table [id=209cd7a3c3344730a035c0590e853779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:48.658679 31080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27417c03d11b400b9ec756ca941432ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:48.658711 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27417c03d11b400b9ec756ca941432ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:48.659027 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27417c03d11b400b9ec756ca941432ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:48.665251 31464 tablet_bootstrap.cc:492] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Bootstrap starting.
I20260808 18:17:48.666381 31467 tablet_bootstrap.cc:492] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Bootstrap starting.
I20260808 18:17:48.667248 31464 tablet_bootstrap.cc:654] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:48.668013 31467 tablet_bootstrap.cc:654] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:48.668093 31464 log.cc:824] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:48.668720 31466 tablet_bootstrap.cc:492] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Bootstrap starting.
I20260808 18:17:48.668854 31467 log.cc:824] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:48.670009 31464 tablet_bootstrap.cc:492] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: No bootstrap required, opened a new log
I20260808 18:17:48.670190 31464 ts_tablet_manager.cc:1397] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 18:17:48.670476 31466 tablet_bootstrap.cc:654] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:48.670641 31467 tablet_bootstrap.cc:492] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: No bootstrap required, opened a new log
I20260808 18:17:48.670787 31467 ts_tablet_manager.cc:1397] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:17:48.671216 31466 log.cc:824] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:48.673136 31466 tablet_bootstrap.cc:492] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: No bootstrap required, opened a new log
I20260808 18:17:48.673347 31466 ts_tablet_manager.cc:1397] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:17:48.673297 31464 raft_consensus.cc:348] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.673561 31464 raft_consensus.cc:374] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:48.673642 31464 raft_consensus.cc:729] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfea4f0710364c53b3bd70fea8b06e22, State: Initialized, Role: FOLLOWER
I20260808 18:17:48.674074 31464 consensus_queue.cc:260] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.674721 31189 heartbeater.cc:503] Master 127.27.18.254:41389 was elected leader, sending a full tablet report...
I20260808 18:17:48.675020 31464 ts_tablet_manager.cc:1428] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 18:17:48.674539 31467 raft_consensus.cc:348] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.675639 31467 raft_consensus.cc:374] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:48.675702 31467 raft_consensus.cc:729] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Initialized, Role: FOLLOWER
I20260808 18:17:48.676128 31467 consensus_queue.cc:260] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.676748 31429 heartbeater.cc:503] Master 127.27.18.254:41389 was elected leader, sending a full tablet report...
I20260808 18:17:48.676878 31466 raft_consensus.cc:348] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.677096 31466 raft_consensus.cc:374] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:48.677150 31466 raft_consensus.cc:729] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Initialized, Role: FOLLOWER
I20260808 18:17:48.677146 31467 ts_tablet_manager.cc:1428] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:17:48.677520 31466 consensus_queue.cc:260] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.678205 30964 heartbeater.cc:503] Master 127.27.18.254:41389 was elected leader, sending a full tablet report...
I20260808 18:17:48.678438 31466 ts_tablet_manager.cc:1428] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 18:17:48.682127 30965 tablet.cc:2369] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:17:48.763923 31473 raft_consensus.cc:482] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:48.764086 31473 raft_consensus.cc:504] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.765110 31473 leader_election.cc:288] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499), dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:48.769573 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.765452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:52657 (local address 127.27.18.195:38499)
0808 18:17:48.765585 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.765592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.765816 (+ 224us) server_negotiation.cc:407] Connection header received
0808 18:17:48.765971 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.765979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.766017 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.766091 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.766351 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.766923 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.768720 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.769078 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.769168 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.769203 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.769372 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.769451 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:17:48.769588 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.765330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:38499 (local address 127.27.18.193:52657)
0808 18:17:48.765721 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.765732 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.765863 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.766103 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.766105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.766128 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.766241 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.766246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.767056 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.767059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.768621 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.768627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.769177 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.769179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.769249 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.769269 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.769353 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.769434 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":225,"thread_start_us":84,"threads_started":1}
I20260808 18:17:48.770262 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27417c03d11b400b9ec756ca941432ed" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2246cfd85da04e38911906e3ac13c208" is_pre_election: true
I20260808 18:17:48.770332 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.765464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:49991 (local address 127.27.18.194:33031)
0808 18:17:48.765601 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.765607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.765747 (+ 140us) server_negotiation.cc:407] Connection header received
0808 18:17:48.765916 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.765919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.765956 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.766027 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.766289 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.766911 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.769247 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.769620 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.769764 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.769839 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.770149 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.770220 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 18:17:48.770337 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.765330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33031 (local address 127.27.18.193:49991)
0808 18:17:48.765664 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.765675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.765799 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.766048 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.766051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.766066 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.766178 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.766183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.767019 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.767023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.768608 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.768619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.769901 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.769905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.770012 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.770031 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.770129 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.770198 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":208,"thread_start_us":74,"threads_started":1}
I20260808 18:17:48.770560 31336 raft_consensus.cc:2454] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 0.
I20260808 18:17:48.770991 30789 leader_election.cc:302] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:48.770980 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27417c03d11b400b9ec756ca941432ed" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22" is_pre_election: true
I20260808 18:17:48.771235 31112 raft_consensus.cc:2454] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 0.
I20260808 18:17:48.771294 31473 raft_consensus.cc:2781] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:48.771387 31473 raft_consensus.cc:482] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:48.771449 31473 raft_consensus.cc:3037] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:48.772579 31473 raft_consensus.cc:504] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.772876 31473 leader_election.cc:288] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 election: Requested vote from peers 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499), dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:48.773192 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27417c03d11b400b9ec756ca941432ed" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22"
I20260808 18:17:48.773232 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27417c03d11b400b9ec756ca941432ed" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2246cfd85da04e38911906e3ac13c208"
I20260808 18:17:48.773332 31112 raft_consensus.cc:3037] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:48.773332 31336 raft_consensus.cc:3037] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:48.774578 31336 raft_consensus.cc:2454] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 1.
I20260808 18:17:48.774578 31112 raft_consensus.cc:2454] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 1.
I20260808 18:17:48.774863 30789 leader_election.cc:302] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:48.775112 31473 raft_consensus.cc:2781] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:48.775264 31473 raft_consensus.cc:686] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 1 LEADER]: Becoming Leader. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Running, Role: LEADER
I20260808 18:17:48.775576 31473 consensus_queue.cc:237] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } }
I20260808 18:17:48.778981 30630 catalog_manager.cc:5495] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9a4e8560bef4465a506278442ac1851 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "f9a4e8560bef4465a506278442ac1851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } health_report { overall_health: HEALTHY } } }
Aug 08 18:17:48 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213067, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:48.828265 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.820700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41389 (local address 127.0.0.1:36062)
0808 18:17:48.820947 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.820959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.821050 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.821254 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.821256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.821581 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:48.821697 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.821702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.822320 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.822322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.822821 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.822828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.823218 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.823220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.823279 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.823783 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:48.823796 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:48.825087 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:48.826605 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.826630 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.826636 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.826907 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.827247 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:48.827250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:48.827251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:48.827336 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:48.827734 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:48.827739 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:48.827832 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.827967 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.828141 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":71,"threads_started":1}
I20260808 18:17:48.828368 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.820829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.27.18.254:41389)
0808 18:17:48.820962 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.820965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.821025 (+ 60us) server_negotiation.cc:407] Connection header received
0808 18:17:48.821121 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.821123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.821156 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.821217 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:48.821794 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.822236 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.822920 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.823128 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.823197 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.825217 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:48.825220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:48.825222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:48.825244 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:48.826483 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.827063 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.827067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.827068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.827153 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:48.827466 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:48.827469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:48.827470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:48.827636 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:48.827718 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.828038 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.828208 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":63}
I20260808 18:17:48.831955 30630 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:36062:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:17:48.832732 30630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:48.837662 30859 tablet_service.cc:1467] Processing CreateTablet for tablet 1f5eacb25aff4a209556e0c1ed258288 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=db5a41a848844dcf945029c26c6967bc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:17:48.837910 31080 tablet_service.cc:1467] Processing CreateTablet for tablet 1f5eacb25aff4a209556e0c1ed258288 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=db5a41a848844dcf945029c26c6967bc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:17:48.837921 31292 tablet_service.cc:1467] Processing CreateTablet for tablet 1f5eacb25aff4a209556e0c1ed258288 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=db5a41a848844dcf945029c26c6967bc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:17:48.838399 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f5eacb25aff4a209556e0c1ed258288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:48.838411 31080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f5eacb25aff4a209556e0c1ed258288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:48.838657 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f5eacb25aff4a209556e0c1ed258288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:48.841369 31464 tablet_bootstrap.cc:492] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: Bootstrap starting.
I20260808 18:17:48.841840 31467 tablet_bootstrap.cc:492] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: Bootstrap starting.
I20260808 18:17:48.842295 31464 tablet_bootstrap.cc:654] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:48.842586 31467 tablet_bootstrap.cc:654] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:48.842891 31466 tablet_bootstrap.cc:492] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: Bootstrap starting.
I20260808 18:17:48.843888 31467 tablet_bootstrap.cc:492] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: No bootstrap required, opened a new log
I20260808 18:17:48.843961 31467 ts_tablet_manager.cc:1397] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:48.844264 31466 tablet_bootstrap.cc:654] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:48.844477 31467 raft_consensus.cc:348] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.844576 31467 raft_consensus.cc:374] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:48.844599 31467 raft_consensus.cc:729] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Initialized, Role: FOLLOWER
I20260808 18:17:48.844744 31467 consensus_queue.cc:260] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.845057 31467 ts_tablet_manager.cc:1428] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:17:48.845003 31472 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.845142 31472 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:48.845551 31464 tablet_bootstrap.cc:492] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: No bootstrap required, opened a new log
I20260808 18:17:48.845561 31466 tablet_bootstrap.cc:492] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: No bootstrap required, opened a new log
I20260808 18:17:48.845645 31464 ts_tablet_manager.cc:1397] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 18:17:48.845670 31466 ts_tablet_manager.cc:1397] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 18:17:48.846128 31464 raft_consensus.cc:348] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.846146 31466 raft_consensus.cc:348] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.846259 31464 raft_consensus.cc:374] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:48.846261 31466 raft_consensus.cc:374] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:48.846293 31466 raft_consensus.cc:729] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Initialized, Role: FOLLOWER
I20260808 18:17:48.846292 31464 raft_consensus.cc:729] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfea4f0710364c53b3bd70fea8b06e22, State: Initialized, Role: FOLLOWER
I20260808 18:17:48.846400 31466 consensus_queue.cc:260] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.846428 31464 consensus_queue.cc:260] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.846671 31464 ts_tablet_manager.cc:1428] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:17:48.846681 31466 ts_tablet_manager.cc:1428] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:17:48.846875 31473 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.847028 31473 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:48.847215 31471 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.847340 31471 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 18:17:48.851125 31430 tablet.cc:2369] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:17:48.865072 31472 raft_consensus.cc:482] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:48.865155 31472 raft_consensus.cc:504] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.866045 31472 leader_election.cc:288] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681), dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:48.870962 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.866302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:54705 (local address 127.27.18.193:38681)
0808 18:17:48.866418 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.866422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.866482 (+ 60us) server_negotiation.cc:407] Connection header received
0808 18:17:48.866641 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.866648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.866685 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.866783 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.867107 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.867916 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.869716 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.870287 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.870400 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.870442 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.870790 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.870859 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:17:48.870962 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.866329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:55855 (local address 127.27.18.194:33031)
0808 18:17:48.866427 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.866432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.866670 (+ 238us) server_negotiation.cc:407] Connection header received
0808 18:17:48.866809 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.866814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.866867 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.866949 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:17:48.867234 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.867966 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.869692 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.870271 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.870394 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.870442 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.870790 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.870859 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:17:48.870995 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.866235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38681 (local address 127.27.18.195:54705)
0808 18:17:48.866409 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.866423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.866519 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.866862 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.866865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.866879 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.867009 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.867019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.868056 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.868060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.869566 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.869571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.870411 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.870415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.870635 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.870686 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.870762 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.870858 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":241}
I20260808 18:17:48.871016 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.866252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33031 (local address 127.27.18.195:55855)
0808 18:17:48.866580 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.866591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.866706 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.866954 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.866958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.866978 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:17:48.867113 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.867121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.868059 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.868061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.869558 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.869569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.870416 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.870419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.870590 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.870685 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.870762 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.870858 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":198,"thread_start_us":94,"threads_started":1}
I20260808 18:17:48.871426 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5eacb25aff4a209556e0c1ed258288" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22" is_pre_election: true
I20260808 18:17:48.871547 31112 raft_consensus.cc:2454] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 0.
I20260808 18:17:48.871755 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5eacb25aff4a209556e0c1ed258288" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a4e8560bef4465a506278442ac1851" is_pre_election: true
I20260808 18:17:48.871882 31225 leader_election.cc:302] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, dfea4f0710364c53b3bd70fea8b06e22; no voters:
I20260808 18:17:48.871982 30890 raft_consensus.cc:2454] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 0.
I20260808 18:17:48.872284 31472 raft_consensus.cc:2781] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:48.872361 31472 raft_consensus.cc:482] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:48.872413 31472 raft_consensus.cc:3037] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:48.873275 31472 raft_consensus.cc:504] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.873631 31472 leader_election.cc:288] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 election: Requested vote from peers f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681), dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:48.873973 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5eacb25aff4a209556e0c1ed258288" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22"
I20260808 18:17:48.873978 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5eacb25aff4a209556e0c1ed258288" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a4e8560bef4465a506278442ac1851"
I20260808 18:17:48.874064 31112 raft_consensus.cc:3037] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:48.874090 30890 raft_consensus.cc:3037] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:48.874881 31112 raft_consensus.cc:2454] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 1.
I20260808 18:17:48.875046 30890 raft_consensus.cc:2454] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 1.
I20260808 18:17:48.875130 31225 leader_election.cc:302] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, dfea4f0710364c53b3bd70fea8b06e22; no voters:
I20260808 18:17:48.875288 31472 raft_consensus.cc:2781] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:48.875542 31472 raft_consensus.cc:686] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [term 1 LEADER]: Becoming Leader. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Running, Role: LEADER
I20260808 18:17:48.875892 31472 consensus_queue.cc:237] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:48.877470 31472 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: TxnStatusTablet state changed. Reason: New leader 2246cfd85da04e38911906e3ac13c208. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.877602 31472 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:17:48.877882 31502 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:17:48.878118 31502 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:17:48.878625 30634 catalog_manager.cc:5495] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2246cfd85da04e38911906e3ac13c208 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } health_report { overall_health: HEALTHY } } }
I20260808 18:17:48.895128 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.891506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:38499 (local address 127.0.0.1:52202)
0808 18:17:48.891682 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.891698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.891806 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.892128 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.892132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.892140 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:48.892269 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.892275 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.892985 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.892990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.893580 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.893587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.894049 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.894053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.894164 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.894206 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.894818 (+ 612us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.894835 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.894928 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.895010 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 18:17:48.895154 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.891560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.27.18.195:38499)
0808 18:17:48.891661 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.891665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.891742 (+ 77us) server_negotiation.cc:407] Connection header received
0808 18:17:48.891953 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.891962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.892008 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.892121 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:48.892387 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.892869 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.893712 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.893932 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.894037 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.894356 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.894721 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.894840 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.894939 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.895009 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 18:17:48.895171 31191 tablet.cc:2369] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:17:48.900362 30890 raft_consensus.cc:1261] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Refusing update from remote peer 2246cfd85da04e38911906e3ac13c208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:48.900364 31112 raft_consensus.cc:1261] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Refusing update from remote peer 2246cfd85da04e38911906e3ac13c208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:48.900900 31472 consensus_queue.cc:1035] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:48.901034 31499 consensus_queue.cc:1035] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:48.903991 31473 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: TxnStatusTablet state changed. Reason: New leader 2246cfd85da04e38911906e3ac13c208. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.904099 31473 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:48.904577 31471 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: TxnStatusTablet state changed. Reason: New leader 2246cfd85da04e38911906e3ac13c208. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.904706 31471 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:48.911340 31499 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.911485 31499 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:17:48.911861 31506 mvcc.cc:205] Tried to move back new op lower bound from 7316328730209611776 to 7316328730118590464. Current Snapshot: MvccSnapshot[applied={T|T < 7316328730209611776}]
I20260808 18:17:48.912101 31499 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.912204 31499 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P 2246cfd85da04e38911906e3ac13c208: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:17:48.913157 31473 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.913257 31473 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P f9a4e8560bef4465a506278442ac1851: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:48.913352 31471 tablet_replica.cc:439] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } } }
I20260808 18:17:48.913439 31471 tablet_replica.cc:442] T 1f5eacb25aff4a209556e0c1ed258288 P dfea4f0710364c53b3bd70fea8b06e22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:17:48.913985 31507 mvcc.cc:205] Tried to move back new op lower bound from 7316328730209611776 to 7316328730118590464. Current Snapshot: MvccSnapshot[applied={T|T < 7316328730209611776}]
I20260808 18:17:48.915963 31509 mvcc.cc:205] Tried to move back new op lower bound from 7316328730209611776 to 7316328730118590464. Current Snapshot: MvccSnapshot[applied={T|T < 7316328730209611776}]
I20260808 18:17:48.928593 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.924514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34346 (local address 127.27.18.193:38681)
0808 18:17:48.924629 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.924634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.924650 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:17:48.924814 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.924817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.924877 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.924986 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:48.925323 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.926093 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.927014 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.927388 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.927495 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.927718 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.928172 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.928283 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.928449 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.928509 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:17:48.939584 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.936150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:45595 (local address 127.27.18.195:38499)
0808 18:17:48.936279 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.936283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.936518 (+ 235us) server_negotiation.cc:407] Connection header received
0808 18:17:48.936682 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.936687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.936733 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.936822 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:48.937071 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.937594 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.938478 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.938717 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.938831 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.939138 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.939220 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.939302 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.939414 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.939461 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:17:48.939630 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.936026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:38499 (local address 127.27.18.193:45595)
0808 18:17:48.936429 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.936441 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.936569 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.936834 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.936837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.936844 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:48.936964 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.936968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.937720 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.937724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.938352 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.938360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.938836 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.938839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.938935 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.938975 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.939312 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.939321 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.939405 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.939480 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":135,"threads_started":1}
I20260808 18:17:48.947006 31112 raft_consensus.cc:1261] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Refusing update from remote peer f9a4e8560bef4465a506278442ac1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:48.947285 31336 raft_consensus.cc:1261] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Refusing update from remote peer f9a4e8560bef4465a506278442ac1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:48.947453 31479 consensus_queue.cc:1035] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:48.947829 31473 consensus_queue.cc:1035] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:48.970155 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.966792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:60645 (local address 127.27.18.193:38681)
0808 18:17:48.966899 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:48.966903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:48.966917 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:17:48.966959 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:48.966964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:48.967007 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:48.967076 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:48.967620 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.968281 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.969090 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.969361 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.969448 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.969746 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.969822 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.969904 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:48.969996 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 18:17:48.970042 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:17:48.970167 31435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:48.966607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38681 (local address 127.27.18.195:60645)
0808 18:17:48.966761 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:48.966775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:48.966848 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:48.967098 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:48.967101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:48.967108 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:48.967244 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.967253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.968417 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.968420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.968971 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:48.968979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:48.969459 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:48.969462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:48.969553 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:48.969599 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:48.969910 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:48.969919 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 18:17:48.969986 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 18:17:48.970062 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 18:17:48.996316 30634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:48.996505 30634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:48.999011 30634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Sending DeleteTablet for 3 replicas of tablet 27417c03d11b400b9ec756ca941432ed
I20260808 18:17:48.999693 31292 tablet_service.cc:1514] Processing DeleteTablet for tablet 27417c03d11b400b9ec756ca941432ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:48 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:52192
I20260808 18:17:48.999861 31080 tablet_service.cc:1514] Processing DeleteTablet for tablet 27417c03d11b400b9ec756ca941432ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:48 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:37930
I20260808 18:17:49.000262 31534 tablet_replica.cc:330] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: stopping tablet replica
I20260808 18:17:49.000551 31534 raft_consensus.cc:2229] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:49.000522 30859 tablet_service.cc:1514] Processing DeleteTablet for tablet 27417c03d11b400b9ec756ca941432ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:48 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:34334
I20260808 18:17:49.000775 31534 raft_consensus.cc:2258] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:49.001116 31537 tablet_replica.cc:330] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: stopping tablet replica
I20260808 18:17:49.001382 31537 raft_consensus.cc:2229] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:17:49.001469 30634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:17:49.001820 30634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:49.001868 31537 raft_consensus.cc:2258] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:49.002112 31534 ts_tablet_manager.cc:1905] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:49.002575 31536 tablet_replica.cc:330] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: stopping tablet replica
I20260808 18:17:49.002912 31536 raft_consensus.cc:2229] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:49.003113 31536 raft_consensus.cc:2258] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:49.004932 31537 ts_tablet_manager.cc:1905] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:49.005450 31534 ts_tablet_manager.cc:1918] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:49.005584 31534 log.cc:1196] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/27417c03d11b400b9ec756ca941432ed
I20260808 18:17:49.005947 31534 ts_tablet_manager.cc:1939] T 27417c03d11b400b9ec756ca941432ed P 2246cfd85da04e38911906e3ac13c208: Deleting consensus metadata
I20260808 18:17:49.007093 30610 catalog_manager.cc:4841] TS 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499): tablet 27417c03d11b400b9ec756ca941432ed (table test-table [id=209cd7a3c3344730a035c0590e853779]) successfully deleted
I20260808 18:17:49.007701 31292 tablet_service.cc:1467] Processing CreateTablet for tablet 14084768f821447292c754ddc20a998d (DEFAULT_TABLE table=test-table [id=e737f73ebd7f4d85acde62a6c4384306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:49.007716 31080 tablet_service.cc:1467] Processing CreateTablet for tablet 14084768f821447292c754ddc20a998d (DEFAULT_TABLE table=test-table [id=e737f73ebd7f4d85acde62a6c4384306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:49.007946 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14084768f821447292c754ddc20a998d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:49.007963 31080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14084768f821447292c754ddc20a998d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:49.008536 31537 ts_tablet_manager.cc:1918] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:49.008700 31537 log.cc:1196] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/27417c03d11b400b9ec756ca941432ed
I20260808 18:17:49.009045 31537 ts_tablet_manager.cc:1939] T 27417c03d11b400b9ec756ca941432ed P f9a4e8560bef4465a506278442ac1851: Deleting consensus metadata
I20260808 18:17:49.010218 31467 tablet_bootstrap.cc:492] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Bootstrap starting.
I20260808 18:17:49.010469 31464 tablet_bootstrap.cc:492] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Bootstrap starting.
I20260808 18:17:49.011178 31467 tablet_bootstrap.cc:654] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:49.011402 31464 tablet_bootstrap.cc:654] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:49.012431 31467 tablet_bootstrap.cc:492] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: No bootstrap required, opened a new log
I20260808 18:17:49.012408 30611 catalog_manager.cc:4841] TS f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681): tablet 27417c03d11b400b9ec756ca941432ed (table test-table [id=209cd7a3c3344730a035c0590e853779]) successfully deleted
I20260808 18:17:49.012516 31467 ts_tablet_manager.cc:1397] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:49.012881 30859 tablet_service.cc:1467] Processing CreateTablet for tablet 14084768f821447292c754ddc20a998d (DEFAULT_TABLE table=test-table [id=e737f73ebd7f4d85acde62a6c4384306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:49.012920 31467 raft_consensus.cc:348] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.013047 31467 raft_consensus.cc:374] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:49.013078 31467 raft_consensus.cc:729] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Initialized, Role: FOLLOWER
I20260808 18:17:49.013136 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14084768f821447292c754ddc20a998d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:49.013190 31467 consensus_queue.cc:260] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.013434 31467 ts_tablet_manager.cc:1428] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:17:49.016340 31466 tablet_bootstrap.cc:492] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Bootstrap starting.
I20260808 18:17:49.017191 31466 tablet_bootstrap.cc:654] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:49.019133 31466 tablet_bootstrap.cc:492] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: No bootstrap required, opened a new log
I20260808 18:17:49.019235 31466 ts_tablet_manager.cc:1397] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 18:17:49.019685 31466 raft_consensus.cc:348] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.019821 31466 raft_consensus.cc:374] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:49.019863 31466 raft_consensus.cc:729] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Initialized, Role: FOLLOWER
I20260808 18:17:49.019986 31466 consensus_queue.cc:260] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.020260 31466 ts_tablet_manager.cc:1428] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:17:49.022230 31536 ts_tablet_manager.cc:1905] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:49.024156 31464 tablet_bootstrap.cc:492] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: No bootstrap required, opened a new log
I20260808 18:17:49.024251 31464 ts_tablet_manager.cc:1397] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260808 18:17:49.024684 31464 raft_consensus.cc:348] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.024820 31464 raft_consensus.cc:374] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:49.024863 31464 raft_consensus.cc:729] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfea4f0710364c53b3bd70fea8b06e22, State: Initialized, Role: FOLLOWER
I20260808 18:17:49.024993 31464 consensus_queue.cc:260] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.025275 31464 ts_tablet_manager.cc:1428] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:17:49.028949 31536 ts_tablet_manager.cc:1918] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:49.029057 31536 log.cc:1196] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/27417c03d11b400b9ec756ca941432ed
I20260808 18:17:49.029394 31536 ts_tablet_manager.cc:1939] T 27417c03d11b400b9ec756ca941432ed P dfea4f0710364c53b3bd70fea8b06e22: Deleting consensus metadata
I20260808 18:17:49.030558 30609 catalog_manager.cc:4841] TS dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031): tablet 27417c03d11b400b9ec756ca941432ed (table test-table [id=209cd7a3c3344730a035c0590e853779]) successfully deleted
I20260808 18:17:49.170867 31529 raft_consensus.cc:482] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:49.170979 31529 raft_consensus.cc:504] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.171478 31529 leader_election.cc:288] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031), 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499)
I20260808 18:17:49.171913 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14084768f821447292c754ddc20a998d" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22" is_pre_election: true
I20260808 18:17:49.171932 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14084768f821447292c754ddc20a998d" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2246cfd85da04e38911906e3ac13c208" is_pre_election: true
I20260808 18:17:49.172075 31112 raft_consensus.cc:2454] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 0.
I20260808 18:17:49.172089 31336 raft_consensus.cc:2454] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 0.
I20260808 18:17:49.172356 30788 leader_election.cc:302] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfea4f0710364c53b3bd70fea8b06e22, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:49.172546 31529 raft_consensus.cc:2781] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:49.172644 31529 raft_consensus.cc:482] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:49.172672 31529 raft_consensus.cc:3037] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:49.173617 31529 raft_consensus.cc:504] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.173938 31529 leader_election.cc:288] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 election: Requested vote from peers dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031), 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499)
I20260808 18:17:49.174247 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14084768f821447292c754ddc20a998d" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22"
I20260808 18:17:49.174355 31112 raft_consensus.cc:3037] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:49.174351 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14084768f821447292c754ddc20a998d" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2246cfd85da04e38911906e3ac13c208"
I20260808 18:17:49.174484 31336 raft_consensus.cc:3037] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:49.175295 31112 raft_consensus.cc:2454] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 1.
I20260808 18:17:49.175477 31336 raft_consensus.cc:2454] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 1.
I20260808 18:17:49.175669 30788 leader_election.cc:302] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfea4f0710364c53b3bd70fea8b06e22, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:49.175869 31529 raft_consensus.cc:2781] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:49.175963 31529 raft_consensus.cc:686] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 1 LEADER]: Becoming Leader. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Running, Role: LEADER
I20260808 18:17:49.176115 31529 consensus_queue.cc:237] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:49.177737 30630 catalog_manager.cc:5495] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9a4e8560bef4465a506278442ac1851 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "f9a4e8560bef4465a506278442ac1851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } health_report { overall_health: UNKNOWN } } }
I20260808 18:17:49.215962 31112 raft_consensus.cc:1261] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Refusing update from remote peer f9a4e8560bef4465a506278442ac1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:49.215962 31336 raft_consensus.cc:1261] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Refusing update from remote peer f9a4e8560bef4465a506278442ac1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:49.216341 31529 consensus_queue.cc:1035] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:49.216478 31473 consensus_queue.cc:1035] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:17:49.228694 31509 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 18:17:50.776907 30634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:50.777045 30634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:50.779349 30634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Sending DeleteTablet for 3 replicas of tablet 14084768f821447292c754ddc20a998d
I20260808 18:17:50.779887 30859 tablet_service.cc:1514] Processing DeleteTablet for tablet 14084768f821447292c754ddc20a998d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:50 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:34334
I20260808 18:17:50.779922 31080 tablet_service.cc:1514] Processing DeleteTablet for tablet 14084768f821447292c754ddc20a998d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:50 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:37930
I20260808 18:17:50.780118 31307 tablet_service.cc:1514] Processing DeleteTablet for tablet 14084768f821447292c754ddc20a998d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:50 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:52192
I20260808 18:17:50.780472 31581 tablet_replica.cc:330] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: stopping tablet replica
I20260808 18:17:50.780545 31584 tablet_replica.cc:330] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: stopping tablet replica
I20260808 18:17:50.780629 31581 raft_consensus.cc:2229] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:17:50.780648 31584 raft_consensus.cc:2229] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:50.780732 31584 raft_consensus.cc:2258] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:50.780802 31581 raft_consensus.cc:2258] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:50.781296 31584 ts_tablet_manager.cc:1905] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:50.781296 31581 ts_tablet_manager.cc:1905] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:50.783874 31584 ts_tablet_manager.cc:1918] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:17:50.783965 31584 log.cc:1196] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/14084768f821447292c754ddc20a998d
I20260808 18:17:50.784184 31583 tablet_replica.cc:330] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: stopping tablet replica
I20260808 18:17:50.784299 31584 ts_tablet_manager.cc:1939] T 14084768f821447292c754ddc20a998d P 2246cfd85da04e38911906e3ac13c208: Deleting consensus metadata
I20260808 18:17:50.784387 31583 raft_consensus.cc:2229] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:50.784509 31583 raft_consensus.cc:2258] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:50.783989 31581 ts_tablet_manager.cc:1918] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:17:50.785032 30610 catalog_manager.cc:4841] TS 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499): tablet 14084768f821447292c754ddc20a998d (table test-table [id=e737f73ebd7f4d85acde62a6c4384306]) successfully deleted
I20260808 18:17:50.785173 31581 log.cc:1196] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/14084768f821447292c754ddc20a998d
I20260808 18:17:50.785451 31581 ts_tablet_manager.cc:1939] T 14084768f821447292c754ddc20a998d P f9a4e8560bef4465a506278442ac1851: Deleting consensus metadata
I20260808 18:17:50.786085 30611 catalog_manager.cc:4841] TS f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681): tablet 14084768f821447292c754ddc20a998d (table test-table [id=e737f73ebd7f4d85acde62a6c4384306]) successfully deleted
I20260808 18:17:50.786115 31583 ts_tablet_manager.cc:1905] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:50.788697 31583 ts_tablet_manager.cc:1918] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:17:50.788756 31583 log.cc:1196] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/14084768f821447292c754ddc20a998d
I20260808 18:17:50.789007 31583 ts_tablet_manager.cc:1939] T 14084768f821447292c754ddc20a998d P dfea4f0710364c53b3bd70fea8b06e22: Deleting consensus metadata
I20260808 18:17:50.789518 30609 catalog_manager.cc:4841] TS dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031): tablet 14084768f821447292c754ddc20a998d (table test-table [id=e737f73ebd7f4d85acde62a6c4384306]) successfully deleted
Aug 08 18:17:50 dist-test-slave-x4qh krb5kdc[30557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:17:50 dist-test-slave-x4qh krb5kdc[30557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:50.806849 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:50.798838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.27.18.254:41389)
0808 18:17:50.799122 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:50.799129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:50.799141 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:17:50.799294 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:50.799297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:50.799337 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:50.799457 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:50.800123 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:50.800631 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:50.801390 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:50.801598 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:50.801669 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:50.803848 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:50.803851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:50.803854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:50.803875 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:50.805490 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:50.805839 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:50.805841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:50.805843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:50.805916 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:50.806130 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:50.806132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:50.806133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:50.806289 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:50.806385 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:50.806538 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 18:17:50.806628 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":119,"threads_started":1}
I20260808 18:17:50.809159 30634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:17:50.809544 30634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:50.814108 31080 tablet_service.cc:1467] Processing CreateTablet for tablet b278c2479810481b83b3e28b61416ff1 (DEFAULT_TABLE table=test-table [id=3a9dcacb7b784a22befbe1fd3cb97153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:50.814131 31307 tablet_service.cc:1467] Processing CreateTablet for tablet b278c2479810481b83b3e28b61416ff1 (DEFAULT_TABLE table=test-table [id=3a9dcacb7b784a22befbe1fd3cb97153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:50.814110 30859 tablet_service.cc:1467] Processing CreateTablet for tablet b278c2479810481b83b3e28b61416ff1 (DEFAULT_TABLE table=test-table [id=3a9dcacb7b784a22befbe1fd3cb97153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:50.814407 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278c2479810481b83b3e28b61416ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:50.814407 31307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278c2479810481b83b3e28b61416ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:50.814438 31080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b278c2479810481b83b3e28b61416ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:50.816951 31613 tablet_bootstrap.cc:492] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Bootstrap starting.
I20260808 18:17:50.817030 31609 tablet_bootstrap.cc:492] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Bootstrap starting.
I20260808 18:17:50.817835 31613 tablet_bootstrap.cc:654] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:50.817932 31609 tablet_bootstrap.cc:654] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:50.818434 31614 tablet_bootstrap.cc:492] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Bootstrap starting.
I20260808 18:17:50.819065 31613 tablet_bootstrap.cc:492] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: No bootstrap required, opened a new log
I20260808 18:17:50.819206 31613 ts_tablet_manager.cc:1397] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:50.819334 31614 tablet_bootstrap.cc:654] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:50.819581 31609 tablet_bootstrap.cc:492] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: No bootstrap required, opened a new log
I20260808 18:17:50.819746 31609 ts_tablet_manager.cc:1397] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:17:50.819720 31613 raft_consensus.cc:348] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.819887 31613 raft_consensus.cc:374] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:50.819933 31613 raft_consensus.cc:729] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Initialized, Role: FOLLOWER
I20260808 18:17:50.820107 31613 consensus_queue.cc:260] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.820274 31609 raft_consensus.cc:348] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.820461 31613 ts_tablet_manager.cc:1428] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:17:50.820529 31609 raft_consensus.cc:374] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:50.820647 31609 raft_consensus.cc:729] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfea4f0710364c53b3bd70fea8b06e22, State: Initialized, Role: FOLLOWER
I20260808 18:17:50.820716 31614 tablet_bootstrap.cc:492] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: No bootstrap required, opened a new log
I20260808 18:17:50.820803 31614 ts_tablet_manager.cc:1397] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 18:17:50.820971 31609 consensus_queue.cc:260] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.821204 31614 raft_consensus.cc:348] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.821399 31609 ts_tablet_manager.cc:1428] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:50.821401 31614 raft_consensus.cc:374] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:50.821455 31614 raft_consensus.cc:729] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Initialized, Role: FOLLOWER
I20260808 18:17:50.821676 31614 consensus_queue.cc:260] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.822014 31614 ts_tablet_manager.cc:1428] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:17:50.977865 31576 raft_consensus.cc:482] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:50.978010 31576 raft_consensus.cc:504] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.978518 31576 leader_election.cc:288] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031), 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499)
I20260808 18:17:50.979076 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b278c2479810481b83b3e28b61416ff1" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2246cfd85da04e38911906e3ac13c208" is_pre_election: true
I20260808 18:17:50.979238 31336 raft_consensus.cc:2454] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 0.
I20260808 18:17:50.979203 31110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b278c2479810481b83b3e28b61416ff1" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22" is_pre_election: true
I20260808 18:17:50.979403 31110 raft_consensus.cc:2454] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 0.
I20260808 18:17:50.979593 30789 leader_election.cc:302] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:50.979779 31576 raft_consensus.cc:2781] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:50.979830 31576 raft_consensus.cc:482] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:50.979862 31576 raft_consensus.cc:3037] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:50.980957 31576 raft_consensus.cc:504] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.981225 31576 leader_election.cc:288] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 election: Requested vote from peers dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031), 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499)
I20260808 18:17:50.981563 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b278c2479810481b83b3e28b61416ff1" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2246cfd85da04e38911906e3ac13c208"
I20260808 18:17:50.981601 31110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b278c2479810481b83b3e28b61416ff1" candidate_uuid: "f9a4e8560bef4465a506278442ac1851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22"
I20260808 18:17:50.981686 31336 raft_consensus.cc:3037] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:50.981714 31110 raft_consensus.cc:3037] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:50.982574 31336 raft_consensus.cc:2454] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 1.
I20260808 18:17:50.982622 31110 raft_consensus.cc:2454] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a4e8560bef4465a506278442ac1851 in term 1.
I20260808 18:17:50.982856 30788 leader_election.cc:302] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfea4f0710364c53b3bd70fea8b06e22, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:50.983063 31576 raft_consensus.cc:2781] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:50.983174 31576 raft_consensus.cc:686] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 1 LEADER]: Becoming Leader. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Running, Role: LEADER
I20260808 18:17:50.983340 31576 consensus_queue.cc:237] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:50.985024 30628 catalog_manager.cc:5495] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9a4e8560bef4465a506278442ac1851 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "f9a4e8560bef4465a506278442ac1851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } health_report { overall_health: UNKNOWN } } }
I20260808 18:17:51.009933 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:51.006983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.27.18.193:38681)
0808 18:17:51.007227 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:51.007233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:51.007259 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:17:51.007296 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:51.007298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:51.007336 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:51.007464 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:51.007677 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.008187 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.008979 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.009188 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.009260 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:51.009493 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:51.009553 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:51.009602 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:51.009681 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 18:17:51.009720 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
I20260808 18:17:51.015244 31336 raft_consensus.cc:1261] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Refusing update from remote peer f9a4e8560bef4465a506278442ac1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:51.015378 31110 raft_consensus.cc:1261] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Refusing update from remote peer f9a4e8560bef4465a506278442ac1851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:51.015744 31529 consensus_queue.cc:1035] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:51.015884 31576 consensus_queue.cc:1035] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:51.044355 30640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:51.044490 30640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:51.046257 30640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Sending DeleteTablet for 3 replicas of tablet b278c2479810481b83b3e28b61416ff1
I20260808 18:17:51.046756 30859 tablet_service.cc:1514] Processing DeleteTablet for tablet b278c2479810481b83b3e28b61416ff1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:51 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:34334
I20260808 18:17:51.046810 31080 tablet_service.cc:1514] Processing DeleteTablet for tablet b278c2479810481b83b3e28b61416ff1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:51 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:37930
I20260808 18:17:51.046865 31307 tablet_service.cc:1514] Processing DeleteTablet for tablet b278c2479810481b83b3e28b61416ff1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:51 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:52192
I20260808 18:17:51.046957 31581 tablet_replica.cc:330] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: stopping tablet replica
I20260808 18:17:51.047017 31583 tablet_replica.cc:330] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: stopping tablet replica
I20260808 18:17:51.047041 31584 tablet_replica.cc:330] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: stopping tablet replica
I20260808 18:17:51.047099 31583 raft_consensus.cc:2229] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:51.047101 31581 raft_consensus.cc:2229] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:17:51.047189 31584 raft_consensus.cc:2229] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:51.047199 31583 raft_consensus.cc:2258] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:51.047262 31581 raft_consensus.cc:2258] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:51.047298 31584 raft_consensus.cc:2258] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:51.048064 31583 ts_tablet_manager.cc:1905] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:51.048064 31584 ts_tablet_manager.cc:1905] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:51.048415 31581 ts_tablet_manager.cc:1905] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:51.050145 31581 ts_tablet_manager.cc:1918] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:51.050192 31581 log.cc:1196] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/b278c2479810481b83b3e28b61416ff1
I20260808 18:17:51.050374 31581 ts_tablet_manager.cc:1939] T b278c2479810481b83b3e28b61416ff1 P f9a4e8560bef4465a506278442ac1851: Deleting consensus metadata
I20260808 18:17:51.050352 31583 ts_tablet_manager.cc:1918] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:51.050352 31584 ts_tablet_manager.cc:1918] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:17:51.050428 31583 log.cc:1196] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/b278c2479810481b83b3e28b61416ff1
I20260808 18:17:51.050428 31584 log.cc:1196] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/b278c2479810481b83b3e28b61416ff1
I20260808 18:17:51.050731 31583 ts_tablet_manager.cc:1939] T b278c2479810481b83b3e28b61416ff1 P dfea4f0710364c53b3bd70fea8b06e22: Deleting consensus metadata
I20260808 18:17:51.050750 31584 ts_tablet_manager.cc:1939] T b278c2479810481b83b3e28b61416ff1 P 2246cfd85da04e38911906e3ac13c208: Deleting consensus metadata
I20260808 18:17:51.050972 30611 catalog_manager.cc:4841] TS f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681): tablet b278c2479810481b83b3e28b61416ff1 (table test-table [id=3a9dcacb7b784a22befbe1fd3cb97153]) successfully deleted
I20260808 18:17:51.051313 30609 catalog_manager.cc:4841] TS dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031): tablet b278c2479810481b83b3e28b61416ff1 (table test-table [id=3a9dcacb7b784a22befbe1fd3cb97153]) successfully deleted
I20260808 18:17:51.051409 30610 catalog_manager.cc:4841] TS 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499): tablet b278c2479810481b83b3e28b61416ff1 (table test-table [id=3a9dcacb7b784a22befbe1fd3cb97153]) successfully deleted
I20260808 18:17:51.051294 30640 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:17:51.051858 30640 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:17:51.055975 30859 tablet_service.cc:1467] Processing CreateTablet for tablet 6fb6f819237f4ab2a875ab5c0f9bc171 (DEFAULT_TABLE table=test-table [id=ed813fcf4a614f739726137de53b0e9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:51.056031 31080 tablet_service.cc:1467] Processing CreateTablet for tablet 6fb6f819237f4ab2a875ab5c0f9bc171 (DEFAULT_TABLE table=test-table [id=ed813fcf4a614f739726137de53b0e9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:51.056245 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb6f819237f4ab2a875ab5c0f9bc171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:51.056205 31307 tablet_service.cc:1467] Processing CreateTablet for tablet 6fb6f819237f4ab2a875ab5c0f9bc171 (DEFAULT_TABLE table=test-table [id=ed813fcf4a614f739726137de53b0e9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:17:51.056300 31080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb6f819237f4ab2a875ab5c0f9bc171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:51.056418 31307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fb6f819237f4ab2a875ab5c0f9bc171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:51.058205 31609 tablet_bootstrap.cc:492] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Bootstrap starting.
I20260808 18:17:51.058507 31613 tablet_bootstrap.cc:492] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851: Bootstrap starting.
I20260808 18:17:51.059134 31609 tablet_bootstrap.cc:654] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:51.059381 31613 tablet_bootstrap.cc:654] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:51.060021 31614 tablet_bootstrap.cc:492] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Bootstrap starting.
I20260808 18:17:51.060305 31609 tablet_bootstrap.cc:492] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: No bootstrap required, opened a new log
I20260808 18:17:51.060360 31609 ts_tablet_manager.cc:1397] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 18:17:51.060539 31613 tablet_bootstrap.cc:492] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851: No bootstrap required, opened a new log
I20260808 18:17:51.060608 31613 ts_tablet_manager.cc:1397] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:51.060647 31609 raft_consensus.cc:348] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.060719 31609 raft_consensus.cc:374] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:51.060739 31609 raft_consensus.cc:729] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfea4f0710364c53b3bd70fea8b06e22, State: Initialized, Role: FOLLOWER
I20260808 18:17:51.060797 31614 tablet_bootstrap.cc:654] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:51.060871 31609 consensus_queue.cc:260] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.061012 31613 raft_consensus.cc:348] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.061112 31613 raft_consensus.cc:374] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:51.061123 31609 ts_tablet_manager.cc:1428] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:17:51.061141 31613 raft_consensus.cc:729] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a4e8560bef4465a506278442ac1851, State: Initialized, Role: FOLLOWER
I20260808 18:17:51.061267 31613 consensus_queue.cc:260] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.061496 31613 ts_tablet_manager.cc:1428] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:17:51.062789 31614 tablet_bootstrap.cc:492] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: No bootstrap required, opened a new log
I20260808 18:17:51.062892 31614 ts_tablet_manager.cc:1397] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 18:17:51.063356 31614 raft_consensus.cc:348] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.063524 31614 raft_consensus.cc:374] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:51.063568 31614 raft_consensus.cc:729] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Initialized, Role: FOLLOWER
I20260808 18:17:51.063694 31614 consensus_queue.cc:260] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.063982 31614 ts_tablet_manager.cc:1428] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 18:17:51.249389 31554 raft_consensus.cc:482] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:17:51.249528 31554 raft_consensus.cc:504] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.249939 31554 leader_election.cc:288] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681), dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:51.250339 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb6f819237f4ab2a875ab5c0f9bc171" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a4e8560bef4465a506278442ac1851" is_pre_election: true
I20260808 18:17:51.250376 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb6f819237f4ab2a875ab5c0f9bc171" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22" is_pre_election: true
I20260808 18:17:51.250542 30890 raft_consensus.cc:2454] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 0.
I20260808 18:17:51.250555 31112 raft_consensus.cc:2454] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 0.
I20260808 18:17:51.250896 31225 leader_election.cc:302] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, dfea4f0710364c53b3bd70fea8b06e22; no voters:
I20260808 18:17:51.251089 31554 raft_consensus.cc:2781] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:17:51.251155 31554 raft_consensus.cc:482] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:17:51.251196 31554 raft_consensus.cc:3037] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:51.252231 31554 raft_consensus.cc:504] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.252552 31554 leader_election.cc:288] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 election: Requested vote from peers f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681), dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031)
I20260808 18:17:51.252859 30890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb6f819237f4ab2a875ab5c0f9bc171" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a4e8560bef4465a506278442ac1851"
I20260808 18:17:51.252923 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fb6f819237f4ab2a875ab5c0f9bc171" candidate_uuid: "2246cfd85da04e38911906e3ac13c208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfea4f0710364c53b3bd70fea8b06e22"
I20260808 18:17:51.252986 30890 raft_consensus.cc:3037] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:51.253059 31112 raft_consensus.cc:3037] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:51.253921 30890 raft_consensus.cc:2454] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 1.
I20260808 18:17:51.253942 31112 raft_consensus.cc:2454] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2246cfd85da04e38911906e3ac13c208 in term 1.
I20260808 18:17:51.254199 31228 leader_election.cc:302] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2246cfd85da04e38911906e3ac13c208, f9a4e8560bef4465a506278442ac1851; no voters:
I20260808 18:17:51.254436 31554 raft_consensus.cc:2781] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:51.254542 31554 raft_consensus.cc:686] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 1 LEADER]: Becoming Leader. State: Replica: 2246cfd85da04e38911906e3ac13c208, State: Running, Role: LEADER
I20260808 18:17:51.254685 31554 consensus_queue.cc:237] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } }
I20260808 18:17:51.256461 30628 catalog_manager.cc:5495] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2246cfd85da04e38911906e3ac13c208 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "2246cfd85da04e38911906e3ac13c208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2246cfd85da04e38911906e3ac13c208" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 38499 } health_report { overall_health: HEALTHY } } }
I20260808 18:17:51.297756 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:51.294885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.27.18.195:38499)
0808 18:17:51.295141 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:51.295155 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:51.295165 (+ 10us) server_negotiation.cc:407] Connection header received
0808 18:17:51.295199 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:51.295201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:51.295237 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:51.295303 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:51.295524 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.296020 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.296751 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.296970 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.297046 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:51.297267 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:51.297350 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:51.297412 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:51.297584 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 18:17:51.297639 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":120,"threads_started":1}
I20260808 18:17:51.306133 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:51.303341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:43819 (local address 127.27.18.195:38499)
0808 18:17:51.303490 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:51.303493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:51.303579 (+ 86us) server_negotiation.cc:407] Connection header received
0808 18:17:51.303671 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:51.303673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:51.303722 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:51.303807 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:17:51.304069 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.304576 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.305302 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.305496 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.305566 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:51.305763 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:17:51.305823 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:17:51.305873 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:51.305964 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 18:17:51.306013 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":69}
I20260808 18:17:51.306146 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:51.303241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:38499 (local address 127.27.18.195:43819)
0808 18:17:51.303501 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:51.303512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:51.303595 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:51.303834 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:51.303836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:51.303842 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:17:51.303978 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:51.303982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.304668 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.304670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:51.305209 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:51.305217 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:51.305589 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:51.305591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:51.305649 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:51.305672 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:17:51.305895 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:17:51.305899 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:17:51.305946 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 18:17:51.306026 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":14,"thread_start_us":88,"threads_started":1}
I20260808 18:17:51.311544 31112 raft_consensus.cc:1261] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Refusing update from remote peer 2246cfd85da04e38911906e3ac13c208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:51.311574 30890 raft_consensus.cc:1261] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851 [term 1 FOLLOWER]: Refusing update from remote peer 2246cfd85da04e38911906e3ac13c208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:17:51.311946 31548 consensus_queue.cc:1035] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfea4f0710364c53b3bd70fea8b06e22" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:17:51.312091 31544 consensus_queue.cc:1035] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a4e8560bef4465a506278442ac1851" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:17:51.324620 31546 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 18:17:52.877259 30628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:52.877404 30628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:17:52.879415 30628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e2d4a9aa81c461495863fb329aefdc4: Sending DeleteTablet for 3 replicas of tablet 6fb6f819237f4ab2a875ab5c0f9bc171
I20260808 18:17:52.879925 30859 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fb6f819237f4ab2a875ab5c0f9bc171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:52 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:34334
I20260808 18:17:52.879971 31080 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fb6f819237f4ab2a875ab5c0f9bc171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:52 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:37930
I20260808 18:17:52.879999 31307 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fb6f819237f4ab2a875ab5c0f9bc171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:17:52 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:52192
I20260808 18:17:52.880384 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30758
I20260808 18:17:52.880424 31678 tablet_replica.cc:330] T 6fb6f819237f4ab2a875ab5c0f9bc171 P f9a4e8560bef4465a506278442ac1851: stopping tablet replica
I20260808 18:17:52.884980 31680 tablet_replica.cc:330] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: stopping tablet replica
I20260808 18:17:52.885205 31680 raft_consensus.cc:2229] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:17:52.885555 31680 raft_consensus.cc:2258] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:52.886596 31680 ts_tablet_manager.cc:1905] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:52.889900 31680 ts_tablet_manager.cc:1918] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:17:52.889962 31680 log.cc:1196] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/6fb6f819237f4ab2a875ab5c0f9bc171
I20260808 18:17:52.890223 31680 ts_tablet_manager.cc:1939] T 6fb6f819237f4ab2a875ab5c0f9bc171 P 2246cfd85da04e38911906e3ac13c208: Deleting consensus metadata
I20260808 18:17:52.890722 31679 tablet_replica.cc:330] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: stopping tablet replica
I20260808 18:17:52.890883 30610 catalog_manager.cc:4841] TS 2246cfd85da04e38911906e3ac13c208 (127.27.18.195:38499): tablet 6fb6f819237f4ab2a875ab5c0f9bc171 (table test-table [id=ed813fcf4a614f739726137de53b0e9e]) successfully deleted
I20260808 18:17:52.890914 31679 raft_consensus.cc:2229] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:17:52.891023 31679 raft_consensus.cc:2258] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:17:52.891609 31679 ts_tablet_manager.cc:1905] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:17:52.893577 31679 ts_tablet_manager.cc:1918] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 18:17:52.893673 31679 log.cc:1196] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/6fb6f819237f4ab2a875ab5c0f9bc171
I20260808 18:17:52.893959 31679 ts_tablet_manager.cc:1939] T 6fb6f819237f4ab2a875ab5c0f9bc171 P dfea4f0710364c53b3bd70fea8b06e22: Deleting consensus metadata
I20260808 18:17:52.894536 30609 catalog_manager.cc:4841] TS dfea4f0710364c53b3bd70fea8b06e22 (127.27.18.194:33031): tablet 6fb6f819237f4ab2a875ab5c0f9bc171 (table test-table [id=ed813fcf4a614f739726137de53b0e9e]) successfully deleted
W20260808 18:17:52.898267 30611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:38681 (error 108)
I20260808 18:17:52.899104 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30975
I20260808 18:17:52.902261 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:52.899082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38681 (local address 127.0.0.1:45834)
0808 18:17:52.902047 (+ 2965us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:52.902136 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:38681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2869,"thread_start_us":96,"threads_started":1}
W20260808 18:17:52.902415 30611 catalog_manager.cc:4584] TS f9a4e8560bef4465a506278442ac1851 (127.27.18.193:38681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fb6f819237f4ab2a875ab5c0f9bc171: Network error: Client connection negotiation failed: client connection to 127.27.18.193:38681: connect: Connection refused (error 111)
I20260808 18:17:52.908908 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 31201
I20260808 18:17:52.920861 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30580
2026-08-08T18:17:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6494 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:52 dist-test-slave-x4qh krb5kdc[31685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:52 dist-test-slave-x4qh krb5kdc[31685](info): set up 2 sockets
Aug 08 18:17:52 dist-test-slave-x4qh krb5kdc[31685](info): commencing operation
krb5kdc: starting...
W20260808 18:17:54.128494 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:54.241115 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:40307
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:40307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:54.346997 31702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:54.347226 31702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:54.347288 31702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:54.351289 31702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:54.351363 31702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:54.351416 31702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:54.351454 31702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:54.351485 31702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:54.356256 31702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:40307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:40307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:54.357097 31702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:54.357939 31702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:54.365821 31712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:54.365769 31707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:54.365849 31709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:54.366011 31702 server_base.cc:1034] running on GCE node
I20260808 18:17:54.366739 31702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:54.367655 31702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:54.368813 31702 hybrid_clock.cc:648] HybridClock initialized: now 1786213074368769 us; error 49 us; skew 500 ppm
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:54.371335 31702 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:54.372284 31702 webserver.cc:460] Webserver started at http://127.27.18.254:38879/ using document root <none> and password file <none>
I20260808 18:17:54.372859 31702 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:54.372951 31702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:54.373167 31702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:54.375015 31702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "463454cec21b4fc3ba4daaac23cd1609"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "4e39b342523600140e31aeb30b6de4af"
server_key_iv: "9aeaadaa89085b4a3cc0eb1278786672"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.375602 31702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "463454cec21b4fc3ba4daaac23cd1609"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "4e39b342523600140e31aeb30b6de4af"
server_key_iv: "9aeaadaa89085b4a3cc0eb1278786672"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.378926 31702 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:17:54.381045 31725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.382089 31702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:54.382234 31702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "463454cec21b4fc3ba4daaac23cd1609"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "4e39b342523600140e31aeb30b6de4af"
server_key_iv: "9aeaadaa89085b4a3cc0eb1278786672"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.382339 31702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:54.415236 31702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:54.417775 31702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:54.417973 31702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:54.424938 31702 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:40307
I20260808 18:17:54.424965 31810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:40307 every 8 connection(s)
I20260808 18:17:54.425796 31702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:54.426671 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 31702
I20260808 18:17:54.426772 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:54.427050 27723 external_mini_cluster.cc:1428] Setting key 64139968781c2a3e241b84992147ce85
I20260808 18:17:54.428776 31811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:54.434820 31811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609: Bootstrap starting.
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:54.436993 31811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:54.437637 31811 log.cc:824] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:54.439356 31811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609: No bootstrap required, opened a new log
I20260808 18:17:54.439903 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:54.428242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.27.18.254:40307)
0808 18:17:54.428611 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:54.428625 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:54.428655 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:17:54.429231 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:54.429246 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:54.429560 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:54.429812 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:54.430699 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.431598 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.432621 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.433019 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.433131 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:54.435751 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:54.435755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:54.435765 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:54.435790 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:54.437265 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:54.437669 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:54.437673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:54.437677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:54.437816 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:54.438079 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:54.438082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:54.438083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:54.438424 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:54.438607 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:54.438882 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 18:17:54.439128 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":129,"threads_started":1}
I20260808 18:17:54.442961 31811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463454cec21b4fc3ba4daaac23cd1609" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40307 } }
I20260808 18:17:54.443138 31811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:54.443220 31811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463454cec21b4fc3ba4daaac23cd1609, State: Initialized, Role: FOLLOWER
I20260808 18:17:54.443667 31811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463454cec21b4fc3ba4daaac23cd1609" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40307 } }
I20260808 18:17:54.443809 31811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:54.443871 31811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:54.443970 31811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:54.444782 31811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463454cec21b4fc3ba4daaac23cd1609" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40307 } }
I20260808 18:17:54.445099 31811 leader_election.cc:302] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 463454cec21b4fc3ba4daaac23cd1609; no voters:
I20260808 18:17:54.445386 31811 leader_election.cc:288] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:54.445547 31823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:54.445776 31823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [term 1 LEADER]: Becoming Leader. State: Replica: 463454cec21b4fc3ba4daaac23cd1609, State: Running, Role: LEADER
I20260808 18:17:54.446076 31823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463454cec21b4fc3ba4daaac23cd1609" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40307 } }
I20260808 18:17:54.446547 31811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:54.447649 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 463454cec21b4fc3ba4daaac23cd1609. Latest consensus state: current_term: 1 leader_uuid: "463454cec21b4fc3ba4daaac23cd1609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463454cec21b4fc3ba4daaac23cd1609" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40307 } } }
I20260808 18:17:54.447650 31824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "463454cec21b4fc3ba4daaac23cd1609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463454cec21b4fc3ba4daaac23cd1609" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40307 } } }
I20260808 18:17:54.447785 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:54.447813 31824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:54.448187 31838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:54.450665 31838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:54.455574 31838 catalog_manager.cc:1347] Generated new cluster ID: 88b862a9740b4bfdb5de0841b6f337cd
I20260808 18:17:54.455627 31838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:54.471712 31838 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:54.472447 31838 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:54.480652 31838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 463454cec21b4fc3ba4daaac23cd1609: Generated new TSK 0
I20260808 18:17:54.481289 31838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:17:54.511917 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:40307
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:54.618587 31856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:54.618868 31856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:54.618942 31856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:54.623095 31856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:54.623158 31856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:54.623263 31856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:17:54.628404 31856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:40307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:54.629344 31856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:54.630321 31856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:54.639640 31867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:54.639649 31865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:54.639708 31856 server_base.cc:1034] running on GCE node
W20260808 18:17:54.639649 31862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:54.640225 31856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:54.640776 31856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:54.641970 31856 hybrid_clock.cc:648] HybridClock initialized: now 1786213074641953 us; error 28 us; skew 500 ppm
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:54.644553 31856 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:17:54.645357 31856 webserver.cc:460] Webserver started at http://127.27.18.193:33485/ using document root <none> and password file <none>
I20260808 18:17:54.645864 31856 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:54.645952 31856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:54.646166 31856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:54.648269 31856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "d9ee8f9462be41b9ae4216319019fb33"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "d41813adda65cceb897b4a5bd9381518"
server_key_iv: "79a5afd5063f3578dacfee61ea9acc3d"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.648878 31856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "d9ee8f9462be41b9ae4216319019fb33"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "d41813adda65cceb897b4a5bd9381518"
server_key_iv: "79a5afd5063f3578dacfee61ea9acc3d"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.652263 31856 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:17:54.654520 31876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.655650 31856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:17:54.655764 31856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "d9ee8f9462be41b9ae4216319019fb33"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "d41813adda65cceb897b4a5bd9381518"
server_key_iv: "79a5afd5063f3578dacfee61ea9acc3d"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.655882 31856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:54.666781 31856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:54.669234 31856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:54.669407 31856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:54.669929 31856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:54.670699 31856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:54.670774 31856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.670838 31856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:54.670879 31856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.679569 31856 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:35411
I20260808 18:17:54.679620 32043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:35411 every 8 connection(s)
I20260808 18:17:54.680339 31856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:17:54.687599 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 31856
I20260808 18:17:54.687721 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:17:54.688073 27723 external_mini_cluster.cc:1428] Setting key fe323987f04fe6c1a3516071f3123f32
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:54.694047 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:54.682009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:43949 (local address 127.27.18.254:40307)
0808 18:17:54.682142 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:54.682146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:54.682882 (+ 736us) server_negotiation.cc:407] Connection header received
0808 18:17:54.684114 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:54.684118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:54.684179 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:54.684256 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:54.685441 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.685970 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.687107 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.687355 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.687506 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:54.690241 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:54.690244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:54.690246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:54.690267 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:54.691838 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:54.692486 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:54.692489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:54.692490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:54.692564 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:54.692974 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:54.692976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:54.692977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:54.693137 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:54.693248 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:54.693742 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 18:17:54.693881 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:17:54.694777 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:54.682329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:40307 (local address 127.27.18.193:43949)
0808 18:17:54.682745 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:54.682782 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:54.683908 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:54.684441 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:54.684447 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:54.684763 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:54.685291 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:54.685312 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.686105 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.686108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:54.686981 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:54.686990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.687542 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.687547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:54.687739 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:54.688349 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:54.688365 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:54.690068 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:54.691976 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:54.691986 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:54.692003 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:54.692371 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:54.692660 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:54.692666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:54.692668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:54.692868 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:54.693251 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:54.693260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:54.693509 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 18:17:54.693695 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 18:17:54.693995 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
I20260808 18:17:54.695962 32046 heartbeater.cc:347] Connected to a master server at 127.27.18.254:40307
I20260808 18:17:54.696173 32046 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:54.696645 32046 heartbeater.cc:511] Master 127.27.18.254:40307 requested a full tablet report, sending...
I20260808 18:17:54.698096 31753 ts_manager.cc:194] Registered new tserver with Master: d9ee8f9462be41b9ae4216319019fb33 (127.27.18.193:35411)
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:17:54.699270 31753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:43949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:17:54.715636 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:40307
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:54.820318 32055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:54.820572 32055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:54.820640 32055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:54.824527 32055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:54.824602 32055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:54.824687 32055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:17:54.829663 32055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:40307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:54.830551 32055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:54.831496 32055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:54.840428 32060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:54.840473 32061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:54.840473 32063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:54.840647 32055 server_base.cc:1034] running on GCE node
I20260808 18:17:54.841027 32055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:54.841578 32055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:54.842684 32055 hybrid_clock.cc:648] HybridClock initialized: now 1786213074842672 us; error 22 us; skew 500 ppm
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:54.845320 32055 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:17:54.846081 32055 webserver.cc:460] Webserver started at http://127.27.18.194:42085/ using document root <none> and password file <none>
I20260808 18:17:54.846647 32055 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:54.846737 32055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:54.847026 32055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:54.848965 32055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "8ec3ebc9e42b4cd995b892ebfc95c38e"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "7973a3f818bf324c83d5669def70de1b"
server_key_iv: "a467c66e0d29d3a8d1aa7bcc3f923588"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.849454 32055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "8ec3ebc9e42b4cd995b892ebfc95c38e"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "7973a3f818bf324c83d5669def70de1b"
server_key_iv: "a467c66e0d29d3a8d1aa7bcc3f923588"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.852862 32055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:17:54.855139 32073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.856261 32055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:17:54.856374 32055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "8ec3ebc9e42b4cd995b892ebfc95c38e"
format_stamp: "Formatted at 2026-08-08 18:17:54 on dist-test-slave-x4qh"
server_key: "7973a3f818bf324c83d5669def70de1b"
server_key_iv: "a467c66e0d29d3a8d1aa7bcc3f923588"
server_key_version: "encryptionkey@0"
I20260808 18:17:54.856451 32055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:54.881067 32055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:54.883827 32055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:54.883961 32055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:54.884442 32055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:54.885231 32055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:54.885275 32055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.885370 32055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:54.885402 32055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:54.894085 32055 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:44413
I20260808 18:17:54.894111 32256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:44413 every 8 connection(s)
I20260808 18:17:54.894886 32055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:17:54.901252 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 32055
I20260808 18:17:54.901343 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:17:54.901615 27723 external_mini_cluster.cc:1428] Setting key 535989d232951866a9ff4cb7c55af431
Aug 08 18:17:54 dist-test-slave-x4qh krb5kdc[31685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213074, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:54.908015 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:54.896519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:45763 (local address 127.27.18.254:40307)
0808 18:17:54.896642 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:54.896645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:54.897360 (+ 715us) server_negotiation.cc:407] Connection header received
0808 18:17:54.898579 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:54.898583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:54.898647 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:54.898751 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:54.899869 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.900384 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.901536 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.901807 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.901919 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:54.904254 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:54.904258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:54.904261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:54.904287 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:54.906053 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:54.906513 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:54.906517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:54.906518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:54.906618 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:54.906989 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:54.906991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:54.906992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:54.907144 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:54.907215 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:54.907814 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 18:17:54.907908 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:17:54.909145 32260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:54.896783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:40307 (local address 127.27.18.194:45763)
0808 18:17:54.897224 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:54.897263 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:54.898377 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:54.898922 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:54.898928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:54.899249 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:54.899722 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:54.899735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.900512 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.900515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:54.901400 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:54.901414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:54.901957 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:54.901962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:54.902126 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:54.902658 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:54.902673 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:54.904099 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:54.906165 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:54.906170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:54.906180 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:54.906417 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:54.906715 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:54.906720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:54.906725 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:54.906899 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:54.907244 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:54.907253 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:54.907567 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 18:17:54.907759 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 18:17:54.908125 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260808 18:17:54.910351 32257 heartbeater.cc:347] Connected to a master server at 127.27.18.254:40307
I20260808 18:17:54.910609 32257 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:54.911213 32257 heartbeater.cc:511] Master 127.27.18.254:40307 requested a full tablet report, sending...
I20260808 18:17:54.912375 31753 ts_manager.cc:194] Registered new tserver with Master: 8ec3ebc9e42b4cd995b892ebfc95c38e (127.27.18.194:44413)
I20260808 18:17:54.912935 31753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:45763
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:17:54.927924 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:40307
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:55.033303 32265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:55.033505 32265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:55.033560 32265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:55.037107 32265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:55.037149 32265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:55.037221 32265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:17:55.041909 32265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:40307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:55.042691 32265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:55.043534 32265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:55.053014 32272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:55.053061 32279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:55.053035 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:55.053223 32265 server_base.cc:1034] running on GCE node
I20260808 18:17:55.053755 32265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:55.054306 32265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:55.055449 32265 hybrid_clock.cc:648] HybridClock initialized: now 1786213075055430 us; error 29 us; skew 500 ppm
Aug 08 18:17:55 dist-test-slave-x4qh krb5kdc[31685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213075, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:55.057974 32265 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:17:55.058784 32265 webserver.cc:460] Webserver started at http://127.27.18.195:37257/ using document root <none> and password file <none>
I20260808 18:17:55.059338 32265 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:55.059458 32265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:55.059636 32265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:55.061479 32265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "a64da287a04049799f871ee19365f841"
format_stamp: "Formatted at 2026-08-08 18:17:55 on dist-test-slave-x4qh"
server_key: "938d4d6664f4008055752039c40231b9"
server_key_iv: "3fc77d24482fbd8584dce068b66bf4b9"
server_key_version: "encryptionkey@0"
I20260808 18:17:55.061978 32265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "a64da287a04049799f871ee19365f841"
format_stamp: "Formatted at 2026-08-08 18:17:55 on dist-test-slave-x4qh"
server_key: "938d4d6664f4008055752039c40231b9"
server_key_iv: "3fc77d24482fbd8584dce068b66bf4b9"
server_key_version: "encryptionkey@0"
I20260808 18:17:55.065299 32265 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:17:55.067444 32288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:55.068500 32265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:17:55.068599 32265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "a64da287a04049799f871ee19365f841"
format_stamp: "Formatted at 2026-08-08 18:17:55 on dist-test-slave-x4qh"
server_key: "938d4d6664f4008055752039c40231b9"
server_key_iv: "3fc77d24482fbd8584dce068b66bf4b9"
server_key_version: "encryptionkey@0"
I20260808 18:17:55.068674 32265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:55.090566 32265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:55.093118 32265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:55.093259 32265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:55.093737 32265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:55.094470 32265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:55.094515 32265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:55.094552 32265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:55.094573 32265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:55.103554 32265 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:41597
I20260808 18:17:55.103601 32480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:41597 every 8 connection(s)
I20260808 18:17:55.104436 32265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
Aug 08 18:17:55 dist-test-slave-x4qh krb5kdc[31685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213075, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:55.113559 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 32265
I20260808 18:17:55.113694 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:17:55.114008 27723 external_mini_cluster.cc:1428] Setting key b9a7674c4ede2aaa7f5f0a13ee281b93
I20260808 18:17:55.117523 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:55.106137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:37555 (local address 127.27.18.254:40307)
0808 18:17:55.106284 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:55.106289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:55.107076 (+ 787us) server_negotiation.cc:407] Connection header received
0808 18:17:55.108218 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:55.108220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:55.108265 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:55.108338 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:55.109349 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:55.109843 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:55.110876 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:55.111076 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:55.111147 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:55.113606 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:55.113612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:55.113617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:55.113683 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:55.115439 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:55.116011 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:55.116032 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:55.116037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:55.116129 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:55.116488 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:55.116492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:55.116494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:55.116671 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:55.116776 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:55.117241 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 18:17:55.117361 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:17:55.118203 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:55.106503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:40307 (local address 127.27.18.195:37555)
0808 18:17:55.106917 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:55.106957 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:55.108031 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:55.108483 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:55.108489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:55.108777 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:55.109223 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:55.109235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:55.109942 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:55.109945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:55.110767 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:55.110775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:55.111196 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:55.111198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:55.111325 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:55.111911 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:55.111926 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:55.113396 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:55.115559 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:55.115564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:55.115578 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:55.115912 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:55.116234 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:55.116238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:55.116240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:55.116383 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:55.116791 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:55.116799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:55.117016 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 18:17:55.117196 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:17:55.117496 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
I20260808 18:17:55.119226 32481 heartbeater.cc:347] Connected to a master server at 127.27.18.254:40307
I20260808 18:17:55.119477 32481 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:55.119915 32481 heartbeater.cc:511] Master 127.27.18.254:40307 requested a full tablet report, sending...
I20260808 18:17:55.120857 31753 ts_manager.cc:194] Registered new tserver with Master: a64da287a04049799f871ee19365f841 (127.27.18.195:41597)
I20260808 18:17:55.121354 31753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:37555
I20260808 18:17:55.127903 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:17:55.141641 31817 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:17:55.140286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.27.18.254:40307)
0808 18:17:55.140416 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:55.140422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:55.140624 (+ 202us) server_negotiation.cc:407] Connection header received
0808 18:17:55.140780 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:55.140783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:55.140838 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:55.140917 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:55.141547 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55480: BlockingRecv error: recv got EOF from 127.0.0.1:55480 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:17:55.142789 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 31856
I20260808 18:17:55.163831 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 32055
I20260808 18:17:55.174088 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 32265
I20260808 18:17:55.181249 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 31702
2026-08-08T18:17:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2258 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:55 dist-test-slave-x4qh krb5kdc[32497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:55 dist-test-slave-x4qh krb5kdc[32497](info): set up 2 sockets
Aug 08 18:17:55 dist-test-slave-x4qh krb5kdc[32497](info): commencing operation
krb5kdc: starting...
W20260808 18:17:56.380394 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:56 dist-test-slave-x4qh krb5kdc[32497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:56.499322 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41005
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:41005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:56.601473 32516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:56.601722 32516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:56.601788 32516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:56.605479 32516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:56.605572 32516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:56.605607 32516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:56.605659 32516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:56.605695 32516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:56.610337 32516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:41005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:56.611169 32516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:56.612047 32516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:56.619695 32527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:56.619882 32526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:56.619938 32516 server_base.cc:1034] running on GCE node
W20260808 18:17:56.619699 32530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:56.620564 32516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:56.621563 32516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:56.622720 32516 hybrid_clock.cc:648] HybridClock initialized: now 1786213076622702 us; error 30 us; skew 500 ppm
Aug 08 18:17:56 dist-test-slave-x4qh krb5kdc[32497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213076, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:56.625538 32516 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:56.626331 32516 webserver.cc:460] Webserver started at http://127.27.18.254:33009/ using document root <none> and password file <none>
I20260808 18:17:56.626848 32516 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:56.626943 32516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:56.627169 32516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:56.629078 32516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "e92b60c62e0b4222a8609729c6e6b60c"
format_stamp: "Formatted at 2026-08-08 18:17:56 on dist-test-slave-x4qh"
server_key: "4b77195055dc232ac765802b4ebc2e19"
server_key_iv: "0421df32cd9aa4a403f0576af5f98e6f"
server_key_version: "encryptionkey@0"
I20260808 18:17:56.629621 32516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "e92b60c62e0b4222a8609729c6e6b60c"
format_stamp: "Formatted at 2026-08-08 18:17:56 on dist-test-slave-x4qh"
server_key: "4b77195055dc232ac765802b4ebc2e19"
server_key_iv: "0421df32cd9aa4a403f0576af5f98e6f"
server_key_version: "encryptionkey@0"
I20260808 18:17:56.633256 32516 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:17:56.635475 32541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:56.636504 32516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:56.636637 32516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "e92b60c62e0b4222a8609729c6e6b60c"
format_stamp: "Formatted at 2026-08-08 18:17:56 on dist-test-slave-x4qh"
server_key: "4b77195055dc232ac765802b4ebc2e19"
server_key_iv: "0421df32cd9aa4a403f0576af5f98e6f"
server_key_version: "encryptionkey@0"
I20260808 18:17:56.636736 32516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:56.649572 32516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:56.652417 32516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:56.652642 32516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:56.659126 32516 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:41005
I20260808 18:17:56.659539 32630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:41005 every 8 connection(s)
I20260808 18:17:56.660099 32516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:56.662582 32631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:56.664793 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 32516
I20260808 18:17:56.664884 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:56.665134 27723 external_mini_cluster.cc:1428] Setting key 615d337a7ff60900ed4faa0164960433
I20260808 18:17:56.668020 32631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c: Bootstrap starting.
I20260808 18:17:56.670442 32631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:56.671319 32631 log.cc:824] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:56.673120 32631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c: No bootstrap required, opened a new log
Aug 08 18:17:56 dist-test-slave-x4qh krb5kdc[32497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:56.676309 32631 raft_consensus.cc:348] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e92b60c62e0b4222a8609729c6e6b60c" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41005 } }
I20260808 18:17:56.676517 32631 raft_consensus.cc:374] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:56.676604 32631 raft_consensus.cc:729] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e92b60c62e0b4222a8609729c6e6b60c, State: Initialized, Role: FOLLOWER
I20260808 18:17:56.677024 32631 consensus_queue.cc:260] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e92b60c62e0b4222a8609729c6e6b60c" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41005 } }
I20260808 18:17:56.677207 32631 raft_consensus.cc:388] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:56.677273 32631 raft_consensus.cc:482] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:56.677371 32631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:17:56.678361 32631 raft_consensus.cc:504] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e92b60c62e0b4222a8609729c6e6b60c" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41005 } }
I20260808 18:17:56.678686 32631 leader_election.cc:302] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e92b60c62e0b4222a8609729c6e6b60c; no voters:
I20260808 18:17:56.679020 32631 leader_election.cc:288] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:56.679121 32641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:56.679163 32635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:56.666283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.27.18.254:41005)
0808 18:17:56.666677 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:56.666708 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:56.666740 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:17:56.667346 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:56.667360 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:56.667707 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:56.667960 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:56.668938 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:56.669800 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:56.670964 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:56.671550 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:56.671741 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:56.674064 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:56.674070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:56.674091 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:56.674116 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:56.676201 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:56.676654 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:56.676661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:56.676667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:56.676797 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:56.677078 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:56.677080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:56.677082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:56.677586 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:56.677760 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:56.678004 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 18:17:56.678274 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":154,"threads_started":1}
I20260808 18:17:56.679399 32641 raft_consensus.cc:686] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [term 1 LEADER]: Becoming Leader. State: Replica: e92b60c62e0b4222a8609729c6e6b60c, State: Running, Role: LEADER
I20260808 18:17:56.679687 32641 consensus_queue.cc:237] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e92b60c62e0b4222a8609729c6e6b60c" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41005 } }
I20260808 18:17:56.680190 32631 sys_catalog.cc:564] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:56.680750 32644 sys_catalog.cc:455] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e92b60c62e0b4222a8609729c6e6b60c. Latest consensus state: current_term: 1 leader_uuid: "e92b60c62e0b4222a8609729c6e6b60c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e92b60c62e0b4222a8609729c6e6b60c" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41005 } } }
I20260808 18:17:56.680876 32644 sys_catalog.cc:458] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:56.681226 32642 sys_catalog.cc:455] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e92b60c62e0b4222a8609729c6e6b60c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e92b60c62e0b4222a8609729c6e6b60c" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41005 } } }
I20260808 18:17:56.681324 32642 sys_catalog.cc:458] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:56.682058 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 18:17:56.684844 32669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:17:56.684927 32669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:17:56.684995 32652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:56.685928 32652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:56.691437 32652 catalog_manager.cc:1347] Generated new cluster ID: 4161cf465add451aad73ee7adf796a9d
I20260808 18:17:56.691556 32652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:56.706364 32635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:56.700959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.27.18.254:41005)
0808 18:17:56.701060 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:56.701065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:56.701255 (+ 190us) server_negotiation.cc:407] Connection header received
0808 18:17:56.701364 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:56.701367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:56.701412 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:56.701499 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:56.702059 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:56.702861 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:56.704047 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:56.704406 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:56.704500 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:56.705401 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:56.705404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:56.705407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:17:56.705901 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:56.705934 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:17:56.705952 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:56.706037 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:56.706163 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 18:17:56.706238 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 18:17:56.707827 32652 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:56.708599 32652 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:56.713440 32652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e92b60c62e0b4222a8609729c6e6b60c: Generated new TSK 0
I20260808 18:17:56.714000 32652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:17:56.719125 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 32516
2026-08-08T18:17:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1539 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:56 dist-test-slave-x4qh krb5kdc[32688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:56 dist-test-slave-x4qh krb5kdc[32688](info): set up 2 sockets
Aug 08 18:17:56 dist-test-slave-x4qh krb5kdc[32688](info): commencing operation
krb5kdc: starting...
W20260808 18:17:57.893467 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:17:57 dist-test-slave-x4qh krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:17:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:17:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:17:58.006108 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:46403
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:46403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:58.112080 32708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:58.112308 32708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:58.112342 32708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:58.116178 32708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:17:58.116221 32708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:58.116240 32708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:58.116253 32708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:17:58.116302 32708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:17:58.121111 32708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:46403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:46403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:58.121989 32708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:58.122869 32708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:58.131014 32722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:58.130976 32721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.131029 32708 server_base.cc:1034] running on GCE node
W20260808 18:17:58.130995 32726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.131781 32708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:58.132798 32708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:58.133960 32708 hybrid_clock.cc:648] HybridClock initialized: now 1786213078133930 us; error 40 us; skew 500 ppm
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:58.139649 32708 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:17:58.140373 32708 webserver.cc:460] Webserver started at http://127.27.18.254:37051/ using document root <none> and password file <none>
I20260808 18:17:58.140851 32708 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:58.140908 32708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:58.141052 32708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:58.142818 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "09ab92a90b874be3a3c41b5d5e718561"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "377abdeed1f3fc5f27e1afab944f4d2c"
server_key_iv: "e34a993edf1566fe8abb5d641a2e7683"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.143307 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "09ab92a90b874be3a3c41b5d5e718561"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "377abdeed1f3fc5f27e1afab944f4d2c"
server_key_iv: "e34a993edf1566fe8abb5d641a2e7683"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.146607 32708 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:17:58.148641 32742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.149709 32708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:58.149801 32708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "09ab92a90b874be3a3c41b5d5e718561"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "377abdeed1f3fc5f27e1afab944f4d2c"
server_key_iv: "e34a993edf1566fe8abb5d641a2e7683"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.149885 32708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:58.162935 32708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:58.165812 32708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:58.165938 32708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:58.172220 32708 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:46403
I20260808 18:17:58.172318 356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:46403 every 8 connection(s)
I20260808 18:17:58.173108 32708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:17:58.175653 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:17:58.180502 358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561: Bootstrap starting.
I20260808 18:17:58.181990 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 32708
I20260808 18:17:58.182086 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:17:58.182399 27723 external_mini_cluster.cc:1428] Setting key 1d5097c4fbd9d6750dcb8581be656706
I20260808 18:17:58.182935 358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561: Neither blocks nor log segments found. Creating new log.
I20260808 18:17:58.183831 358 log.cc:824] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561: Log is configured to *not* fsync() on all Append() calls
I20260808 18:17:58.185887 358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561: No bootstrap required, opened a new log
I20260808 18:17:58.188938 358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ab92a90b874be3a3c41b5d5e718561" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46403 } }
I20260808 18:17:58.189169 358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:17:58.189242 358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09ab92a90b874be3a3c41b5d5e718561, State: Initialized, Role: FOLLOWER
I20260808 18:17:58.189695 358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ab92a90b874be3a3c41b5d5e718561" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46403 } }
I20260808 18:17:58.189818 358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:17:58.189898 358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:17:58.190012 358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:58.190927 358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ab92a90b874be3a3c41b5d5e718561" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46403 } }
I20260808 18:17:58.191249 358 leader_election.cc:302] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09ab92a90b874be3a3c41b5d5e718561; no voters:
I20260808 18:17:58.191609 358 leader_election.cc:288] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:17:58.191684 366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:17:58.191902 366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [term 1 LEADER]: Becoming Leader. State: Replica: 09ab92a90b874be3a3c41b5d5e718561, State: Running, Role: LEADER
I20260808 18:17:58.192180 366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ab92a90b874be3a3c41b5d5e718561" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46403 } }
I20260808 18:17:58.192842 358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:17:58.193313 369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09ab92a90b874be3a3c41b5d5e718561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ab92a90b874be3a3c41b5d5e718561" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46403 } } }
I20260808 18:17:58.193452 369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:58.194132 368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09ab92a90b874be3a3c41b5d5e718561. Latest consensus state: current_term: 1 leader_uuid: "09ab92a90b874be3a3c41b5d5e718561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ab92a90b874be3a3c41b5d5e718561" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46403 } } }
I20260808 18:17:58.194234 368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561 [sys.catalog]: This master's current role is: LEADER
I20260808 18:17:58.196764 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.183424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.27.18.254:46403)
0808 18:17:58.183901 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:58.183921 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:58.183961 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:17:58.184718 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:58.184740 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:58.185100 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:58.185437 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:58.186117 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.187056 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.187899 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.188339 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.188496 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.191083 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:58.191091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:58.191106 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:58.191135 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:58.193206 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.193719 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.193728 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.193737 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.193903 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.194269 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.194272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.194274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.194806 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:58.195023 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:58.195700 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 18:17:58.195979 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":130,"threads_started":1}
W20260808 18:17:58.197471 392 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:17:58.197556 392 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:17:58.198366 374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:17:58.199079 374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:17:58.204198 374 catalog_manager.cc:1347] Generated new cluster ID: 1a30a8f0a4d344f787586ed85ad3f9f3
I20260808 18:17:58.204262 374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:17:58.220330 374 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:17:58.221416 374 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:17:58.229398 374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09ab92a90b874be3a3c41b5d5e718561: Generated new TSK 0
I20260808 18:17:58.229996 374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:17:58.259734 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46403
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:58.368152 403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:58.368386 403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:58.368429 403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:58.372219 403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:58.372265 403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:58.372339 403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:17:58.377274 403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:58.378120 403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:58.378990 403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:58.388238 413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:58.388315 414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.388303 403 server_base.cc:1034] running on GCE node
W20260808 18:17:58.388221 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.388811 403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:58.389494 403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:58.390661 403 hybrid_clock.cc:648] HybridClock initialized: now 1786213078390632 us; error 38 us; skew 500 ppm
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:58.393461 403 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:17:58.394435 403 webserver.cc:460] Webserver started at http://127.27.18.193:35083/ using document root <none> and password file <none>
I20260808 18:17:58.394999 403 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:58.395123 403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:58.395294 403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:58.397161 403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "39fa02e551b44b23942a37a0daea1f04"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "74a69708f31088fa5ea44ebfe17dc99e"
server_key_iv: "38abf32362816dd8381217dfb5cf997d"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.397648 403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "39fa02e551b44b23942a37a0daea1f04"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "74a69708f31088fa5ea44ebfe17dc99e"
server_key_iv: "38abf32362816dd8381217dfb5cf997d"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.401108 403 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:17:58.403270 427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.404510 403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:17:58.404644 403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "39fa02e551b44b23942a37a0daea1f04"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "74a69708f31088fa5ea44ebfe17dc99e"
server_key_iv: "38abf32362816dd8381217dfb5cf997d"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.404769 403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:58.424403 403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:58.427338 403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:58.427533 403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:58.428085 403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:58.428808 403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:58.428849 403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.428908 403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:58.428936 403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.437942 403 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:37109
I20260808 18:17:58.437963 614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:37109 every 8 connection(s)
I20260808 18:17:58.438776 403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:17:58.445402 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 403
I20260808 18:17:58.445546 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:17:58.445842 27723 external_mini_cluster.cc:1428] Setting key 5e8cbd22d93aa2d0748e6495cb57e3b4
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:58.452688 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.440489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:44107 (local address 127.27.18.254:46403)
0808 18:17:58.440625 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:58.440629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:58.441377 (+ 748us) server_negotiation.cc:407] Connection header received
0808 18:17:58.442815 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:58.442818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:58.442876 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:58.442974 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:58.444144 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.444676 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.445962 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.446244 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.446369 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.448848 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:58.448854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:58.448859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:58.448890 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:58.450256 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.450951 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.450956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.450958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.451045 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.451431 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.451435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.451438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.451625 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:58.451754 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:58.452310 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 18:17:58.452447 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:17:58.453498 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.440814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46403 (local address 127.27.18.193:44107)
0808 18:17:58.441241 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:58.441279 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:58.442598 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:58.443146 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:58.443153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:58.443536 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:58.443993 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:58.444006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.444810 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.444818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:58.445829 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:58.445839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.446430 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.446434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:58.446627 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.447193 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:58.447224 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:58.448694 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:58.450396 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:58.450404 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:58.450421 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:58.450834 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:58.451157 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:58.451160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:58.451162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:58.451299 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:58.451755 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:58.451763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:58.451993 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 18:17:58.452263 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 18:17:58.452574 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":123,"threads_started":1}
I20260808 18:17:58.454885 615 heartbeater.cc:347] Connected to a master server at 127.27.18.254:46403
I20260808 18:17:58.455116 615 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:58.455636 615 heartbeater.cc:511] Master 127.27.18.254:46403 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:17:58.457158 302 ts_manager.cc:194] Registered new tserver with Master: 39fa02e551b44b23942a37a0daea1f04 (127.27.18.193:37109)
I20260808 18:17:58.458369 302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:44107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:17:58.473907 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46403
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:58.578053 624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:58.578284 624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:58.578329 624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:58.582546 624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:58.582594 624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:58.582644 624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:17:58.587457 624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:58.588186 624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:58.588940 624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:58.597651 633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:58.597666 632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.597836 624 server_base.cc:1034] running on GCE node
W20260808 18:17:58.597651 637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.598425 624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:58.598934 624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:58.600083 624 hybrid_clock.cc:648] HybridClock initialized: now 1786213078600067 us; error 27 us; skew 500 ppm
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:58.602694 624 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:17:58.603541 624 webserver.cc:460] Webserver started at http://127.27.18.194:38775/ using document root <none> and password file <none>
I20260808 18:17:58.604074 624 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:58.604164 624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:58.604405 624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:58.606276 624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "27a18d7908864871bfd448987dec9782"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "957115788b3d9380d99ce8f1d462b3bb"
server_key_iv: "fe27a907a13f292fe8099099354f50da"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.606828 624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "27a18d7908864871bfd448987dec9782"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "957115788b3d9380d99ce8f1d462b3bb"
server_key_iv: "fe27a907a13f292fe8099099354f50da"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.610493 624 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 18:17:58.612882 654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.614050 624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:17:58.614204 624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "27a18d7908864871bfd448987dec9782"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "957115788b3d9380d99ce8f1d462b3bb"
server_key_iv: "fe27a907a13f292fe8099099354f50da"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.614317 624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:58.634990 624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:58.637953 624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:58.638149 624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:58.638684 624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:58.639505 624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:58.639577 624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.639650 624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:58.639701 624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.648590 624 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:39337
I20260808 18:17:58.648628 835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:39337 every 8 connection(s)
I20260808 18:17:58.649400 624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:17:58.649652 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 624
I20260808 18:17:58.649791 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:17:58.650122 27723 external_mini_cluster.cc:1428] Setting key bf5b3f52a117b9aaf3b6c2dbfe489991
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
I20260808 18:17:58.663421 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.651258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:44013 (local address 127.27.18.254:46403)
0808 18:17:58.651419 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:58.651423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:58.652343 (+ 920us) server_negotiation.cc:407] Connection header received
0808 18:17:58.653918 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:58.653920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:58.653964 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:58.654078 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:58.655185 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.655780 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.656873 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.657084 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.657191 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.659855 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:58.659859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:58.659861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:58.659895 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:58.661270 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.661773 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.661776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.661777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.661853 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.662140 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.662142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.662143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.662385 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:58.662464 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:58.663088 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 18:17:58.663186 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:17:58.664204 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.651732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46403 (local address 127.27.18.194:44013)
0808 18:17:58.652182 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:58.652233 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:58.653625 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:58.654259 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:58.654265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:58.654598 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:58.655043 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:58.655055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.655911 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.655914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:58.656747 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:58.656756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.657230 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.657234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:58.657408 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.657958 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:58.657974 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:58.659680 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:58.661429 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:58.661434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:58.661444 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:58.661682 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:58.661936 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:58.661938 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:58.661939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:58.662059 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:58.662464 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:58.662469 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:58.662714 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 18:17:58.662961 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 18:17:58.663325 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":137,"threads_started":1}
I20260808 18:17:58.665374 837 heartbeater.cc:347] Connected to a master server at 127.27.18.254:46403
I20260808 18:17:58.665612 837 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:58.666201 837 heartbeater.cc:511] Master 127.27.18.254:46403 requested a full tablet report, sending...
I20260808 18:17:58.667440 302 ts_manager.cc:194] Registered new tserver with Master: 27a18d7908864871bfd448987dec9782 (127.27.18.194:39337)
I20260808 18:17:58.668108 302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:44013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:17:58.678570 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46403
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:17:58.783883 845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:17:58.784121 845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:17:58.784192 845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:17:58.787817 845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:17:58.787879 845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:17:58.787951 845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:17:58.792136 845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:17:58.792920 845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:17:58.793848 845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:17:58.802448 850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.802559 845 server_base.cc:1034] running on GCE node
W20260808 18:17:58.802495 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:17:58.802448 853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:17:58.803085 845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:17:58.803736 845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:17:58.804914 845 hybrid_clock.cc:648] HybridClock initialized: now 1786213078804898 us; error 28 us; skew 500 ppm
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:17:58.807724 845 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:17:58.808524 845 webserver.cc:460] Webserver started at http://127.27.18.195:34503/ using document root <none> and password file <none>
I20260808 18:17:58.809106 845 fs_manager.cc:359] Metadata directory not provided
I20260808 18:17:58.809196 845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:17:58.809427 845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:17:58.811233 845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "9e57e5780ca9403db9717ecf4788a076"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "440030882e87ca9484776691e57cad72"
server_key_iv: "388d5766e657331bd382a530be89dd4f"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.811823 845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "9e57e5780ca9403db9717ecf4788a076"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "440030882e87ca9484776691e57cad72"
server_key_iv: "388d5766e657331bd382a530be89dd4f"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.815442 845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:17:58.817698 862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.818849 845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:17:58.818988 845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "9e57e5780ca9403db9717ecf4788a076"
format_stamp: "Formatted at 2026-08-08 18:17:58 on dist-test-slave-x4qh"
server_key: "440030882e87ca9484776691e57cad72"
server_key_iv: "388d5766e657331bd382a530be89dd4f"
server_key_version: "encryptionkey@0"
I20260808 18:17:58.819093 845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:17:58.832180 845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:17:58.835561 845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:17:58.835738 845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:17:58.836275 845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:17:58.837064 845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:17:58.837136 845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.837198 845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:17:58.837250 845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:17:58.845827 845 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:35649
I20260808 18:17:58.845922 1045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:35649 every 8 connection(s)
I20260808 18:17:58.846698 845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:17:58.854266 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 845
I20260808 18:17:58.854398 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:17:58.854687 27723 external_mini_cluster.cc:1428] Setting key 6e2a1aa204ade0beae5d4cbbcf568758
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:58.859440 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.848331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:43281 (local address 127.27.18.254:46403)
0808 18:17:58.848461 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:58.848465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:58.849248 (+ 783us) server_negotiation.cc:407] Connection header received
0808 18:17:58.850535 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:58.850538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:58.850592 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:58.850689 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:58.851688 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.852199 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.853284 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.853570 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.853707 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.856225 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:58.856229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:58.856232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:58.856274 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:58.857521 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.858024 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.858028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.858030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.858112 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.858451 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.858455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.858456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.858631 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:58.858734 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:58.859146 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 18:17:58.859253 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:17:58.860033 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.848613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46403 (local address 127.27.18.195:43281)
0808 18:17:58.849099 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 18:17:58.849135 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:17:58.850316 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:17:58.850849 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:17:58.850856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:17:58.851149 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:17:58.851553 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:58.851564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.852324 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.852328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:58.853156 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:17:58.853166 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.853744 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.853748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:17:58.853914 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.854475 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:17:58.854516 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:17:58.856067 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:17:58.857646 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:58.857651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:58.857664 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:58.857919 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:58.858211 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:17:58.858214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:17:58.858216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:17:58.858362 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:17:58.858734 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:17:58.858741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:17:58.858939 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 18:17:58.859105 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 18:17:58.859362 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":108,"threads_started":1}
I20260808 18:17:58.860994 1046 heartbeater.cc:347] Connected to a master server at 127.27.18.254:46403
I20260808 18:17:58.861178 1046 heartbeater.cc:464] Registering TS with master...
I20260808 18:17:58.861548 1046 heartbeater.cc:511] Master 127.27.18.254:46403 requested a full tablet report, sending...
I20260808 18:17:58.862412 302 ts_manager.cc:194] Registered new tserver with Master: 9e57e5780ca9403db9717ecf4788a076 (127.27.18.195:35649)
I20260808 18:17:58.862959 302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:43281
I20260808 18:17:58.868109 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[32688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213078, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:17:58.892598 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:17:58.885201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48160 (local address 127.27.18.254:46403)
0808 18:17:58.885343 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:17:58.885349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:17:58.885544 (+ 195us) server_negotiation.cc:407] Connection header received
0808 18:17:58.885622 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:17:58.885624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:17:58.885658 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:17:58.885718 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:17:58.886311 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.886749 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.887605 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:17:58.887796 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:17:58.887869 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:17:58.889967 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:17:58.889971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:17:58.889972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:17:58.889992 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:17:58.891051 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.891454 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.891458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.891460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.891533 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:17:58.891790 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:17:58.891793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:17:58.891795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:17:58.891980 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:17:58.892113 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:17:58.892366 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 18:17:58.892470 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 18:17:58.892925 302 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48160
I20260808 18:17:58.893779 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 403
I20260808 18:17:58.909519 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 624
I20260808 18:17:58.929105 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 845
I20260808 18:17:58.940142 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 32708
2026-08-08T18:17:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2219 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[1067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[1067](info): set up 2 sockets
Aug 08 18:17:58 dist-test-slave-x4qh krb5kdc[1067](info): commencing operation
krb5kdc: starting...
W20260808 18:18:00.112751 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:18:00.226123 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42473
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:00.332304 1091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:00.332516 1091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:00.332551 1091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:00.336304 1091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:00.336350 1091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:00.336369 1091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:00.336386 1091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:00.336401 1091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:00.341104 1091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:00.341970 1091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:00.342833 1091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:00.350632 1104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:00.350652 1101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:00.350735 1091 server_base.cc:1034] running on GCE node
W20260808 18:18:00.350652 1108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:00.351526 1091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:00.352468 1091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:00.353618 1091 hybrid_clock.cc:648] HybridClock initialized: now 1786213080353601 us; error 28 us; skew 500 ppm
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:00.356345 1091 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:18:00.357110 1091 webserver.cc:460] Webserver started at http://127.27.18.254:35635/ using document root <none> and password file <none>
I20260808 18:18:00.357601 1091 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:00.357685 1091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:00.357893 1091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:00.359772 1091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "8834aeaa48e34882811acd3752bb1a94"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "f8d07cf71f736558d4cbbfc533bb93e9"
server_key_iv: "70c5f66d5b5555ea17727abe2fcfd7cd"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.360316 1091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "8834aeaa48e34882811acd3752bb1a94"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "f8d07cf71f736558d4cbbfc533bb93e9"
server_key_iv: "70c5f66d5b5555ea17727abe2fcfd7cd"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.363798 1091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:00.366091 1119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.367180 1091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:00.367321 1091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "8834aeaa48e34882811acd3752bb1a94"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "f8d07cf71f736558d4cbbfc533bb93e9"
server_key_iv: "70c5f66d5b5555ea17727abe2fcfd7cd"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.367486 1091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:00.382925 1091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:00.385582 1091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:00.385756 1091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:00.392457 1091 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:42473
I20260808 18:18:00.392485 1208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:42473 every 8 connection(s)
I20260808 18:18:00.393245 1091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:00.395767 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:00.400664 1209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94: Bootstrap starting.
I20260808 18:18:00.402122 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 1091
I20260808 18:18:00.402232 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:00.402487 27723 external_mini_cluster.cc:1428] Setting key d2fa56dd35594f72fee195ef1991b9c3
I20260808 18:18:00.403090 1209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:00.403898 1209 log.cc:824] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:00.405963 1209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94: No bootstrap required, opened a new log
I20260808 18:18:00.409224 1209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8834aeaa48e34882811acd3752bb1a94" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42473 } }
I20260808 18:18:00.409404 1209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:00.409475 1209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8834aeaa48e34882811acd3752bb1a94, State: Initialized, Role: FOLLOWER
I20260808 18:18:00.409870 1209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8834aeaa48e34882811acd3752bb1a94" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42473 } }
I20260808 18:18:00.409983 1209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:00.410066 1209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:00.410166 1209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:00.411017 1209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8834aeaa48e34882811acd3752bb1a94" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42473 } }
I20260808 18:18:00.411350 1209 leader_election.cc:302] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8834aeaa48e34882811acd3752bb1a94; no voters:
I20260808 18:18:00.411761 1209 leader_election.cc:288] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:00.411887 1218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:00.412110 1218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [term 1 LEADER]: Becoming Leader. State: Replica: 8834aeaa48e34882811acd3752bb1a94, State: Running, Role: LEADER
I20260808 18:18:00.412493 1218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8834aeaa48e34882811acd3752bb1a94" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42473 } }
I20260808 18:18:00.412981 1209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:00.414223 1219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8834aeaa48e34882811acd3752bb1a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8834aeaa48e34882811acd3752bb1a94" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42473 } } }
I20260808 18:18:00.414357 1219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:00.414742 1220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8834aeaa48e34882811acd3752bb1a94. Latest consensus state: current_term: 1 leader_uuid: "8834aeaa48e34882811acd3752bb1a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8834aeaa48e34882811acd3752bb1a94" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42473 } } }
I20260808 18:18:00.415416 1220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:00.416981 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:00.403541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42074 (local address 127.27.18.254:42473)
0808 18:18:00.403981 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:00.404001 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:00.404057 (+ 56us) server_negotiation.cc:407] Connection header received
0808 18:18:00.404807 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:00.404832 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:00.405186 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:00.405526 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:00.406225 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.407140 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.408065 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.408505 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.408683 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:00.411243 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:00.411248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:00.411259 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:00.411284 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:00.413452 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:00.414022 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:00.414028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:00.414034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:00.414205 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:00.414557 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:00.414561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:00.414564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:00.415103 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:00.415325 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:00.416101 (+ 776us) server_negotiation.cc:299] Negotiation successful
0808 18:18:00.416326 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":122,"threads_started":1}
W20260808 18:18:00.419310 1241 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:18:00.419456 1241 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:18:00.419560 1233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:00.420261 1233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:00.425611 1233 catalog_manager.cc:1347] Generated new cluster ID: be3f2f80a3914d02ad6c759cd56a4c27
I20260808 18:18:00.425678 1233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:00.438196 1233 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:00.439232 1233 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:00.444988 1233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8834aeaa48e34882811acd3752bb1a94: Generated new TSK 0
I20260808 18:18:00.445613 1233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:18:00.474046 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42473
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:00.579488 1248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:00.579737 1248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:00.579811 1248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:00.583648 1248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:00.583711 1248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:00.583786 1248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:00.588379 1248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:00.589282 1248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:00.590150 1248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:00.598708 1256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:00.598735 1255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:00.598763 1259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:00.599011 1248 server_base.cc:1034] running on GCE node
I20260808 18:18:00.599498 1248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:00.600024 1248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:00.601194 1248 hybrid_clock.cc:648] HybridClock initialized: now 1786213080601173 us; error 43 us; skew 500 ppm
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:00.603724 1248 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:18:00.604513 1248 webserver.cc:460] Webserver started at http://127.27.18.193:34803/ using document root <none> and password file <none>
I20260808 18:18:00.605020 1248 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:00.605106 1248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:00.605329 1248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:00.607143 1248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "e00be765e42348ac843e042adc1d0575"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "7d06fc39d4b7fb7371f688efee07cae7"
server_key_iv: "46d26f7663401070de7754f1e1442923"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.607725 1248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "e00be765e42348ac843e042adc1d0575"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "7d06fc39d4b7fb7371f688efee07cae7"
server_key_iv: "46d26f7663401070de7754f1e1442923"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.611286 1248 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:00.613587 1271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.615144 1248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:18:00.615235 1248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "e00be765e42348ac843e042adc1d0575"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "7d06fc39d4b7fb7371f688efee07cae7"
server_key_iv: "46d26f7663401070de7754f1e1442923"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.615316 1248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:00.623351 1248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:00.625957 1248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:00.626092 1248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:00.626556 1248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:00.627249 1248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:00.627292 1248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.627329 1248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:00.627351 1248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.635489 1248 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:38159
I20260808 18:18:00.635510 1450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:38159 every 8 connection(s)
I20260808 18:18:00.636273 1248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:00.639873 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 1248
I20260808 18:18:00.639995 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:00.640281 27723 external_mini_cluster.cc:1428] Setting key 572cd613fe9dd1595bdca2c5c42de0cd
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:18:00.650569 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:00.637960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:54079 (local address 127.27.18.254:42473)
0808 18:18:00.638110 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:00.638115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:00.638947 (+ 832us) server_negotiation.cc:407] Connection header received
0808 18:18:00.640618 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:00.640621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:00.640697 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:00.640792 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:00.642113 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.642631 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.644277 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.644535 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.644653 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:00.647206 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:00.647209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:00.647211 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:00.647232 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:00.648633 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:00.649117 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:00.649121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:00.649123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:00.649203 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:00.649533 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:00.649537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:00.649539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:00.649716 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:00.649838 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:00.650286 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 18:18:00.650407 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:18:00.651257 1455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:00.638387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42473 (local address 127.27.18.193:54079)
0808 18:18:00.638812 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:00.638850 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:00.640414 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:00.640973 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:00.640981 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:00.641315 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:00.641963 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:00.641985 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.642764 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.642771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:00.644142 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:00.644153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.644691 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.644695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:00.644868 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:00.645401 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:00.645427 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:00.647041 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:00.648753 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:00.648758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:00.648771 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:00.649015 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:00.649312 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:00.649315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:00.649317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:00.649442 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:00.649832 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:00.649838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:00.650071 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 18:18:00.650243 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 18:18:00.650532 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260808 18:18:00.652310 1452 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42473
I20260808 18:18:00.652489 1452 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:18:00.652946 1452 heartbeater.cc:511] Master 127.27.18.254:42473 requested a full tablet report, sending...
I20260808 18:18:00.654554 1149 ts_manager.cc:194] Registered new tserver with Master: e00be765e42348ac843e042adc1d0575 (127.27.18.193:38159)
I20260808 18:18:00.655726 1149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:54079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:18:00.667620 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42473
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:00.773368 1458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:00.773576 1458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:00.773629 1458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:00.777540 1458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:00.777582 1458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:00.777650 1458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:00.782287 1458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:00.783129 1458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:00.784428 1458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:00.793411 1471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:00.793579 1458 server_base.cc:1034] running on GCE node
W20260808 18:18:00.793419 1466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:00.793411 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:00.794086 1458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:00.794590 1458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:00.795742 1458 hybrid_clock.cc:648] HybridClock initialized: now 1786213080795710 us; error 43 us; skew 500 ppm
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:00.798178 1458 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:18:00.798841 1458 webserver.cc:460] Webserver started at http://127.27.18.194:44711/ using document root <none> and password file <none>
I20260808 18:18:00.799263 1458 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:00.799317 1458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:00.799527 1458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:00.801229 1458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "14236ba9d7894f05bcc72607f64a432e"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "c8c84aae139c5d8fc6ba633ef44afc69"
server_key_iv: "4db986d6bb7f497fd34fe85fa9cc44c3"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.801694 1458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "14236ba9d7894f05bcc72607f64a432e"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "c8c84aae139c5d8fc6ba633ef44afc69"
server_key_iv: "4db986d6bb7f497fd34fe85fa9cc44c3"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.805012 1458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:00.807078 1485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.808185 1458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:00.808305 1458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "14236ba9d7894f05bcc72607f64a432e"
format_stamp: "Formatted at 2026-08-08 18:18:00 on dist-test-slave-x4qh"
server_key: "c8c84aae139c5d8fc6ba633ef44afc69"
server_key_iv: "4db986d6bb7f497fd34fe85fa9cc44c3"
server_key_version: "encryptionkey@0"
I20260808 18:18:00.808403 1458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:00.820299 1458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:00.823252 1458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:00.823463 1458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:00.824002 1458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:00.824756 1458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:00.824800 1458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.824872 1458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:00.824908 1458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:00.833097 1458 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:37207
I20260808 18:18:00.833150 1654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:37207 every 8 connection(s)
I20260808 18:18:00.833894 1458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:00.843287 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 1458
I20260808 18:18:00.843436 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:00.843729 27723 external_mini_cluster.cc:1428] Setting key e2e2608439b677a5ec904914de60d643
I20260808 18:18:00.847031 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:00.835523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:36717 (local address 127.27.18.254:42473)
0808 18:18:00.835651 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:00.835655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:00.836401 (+ 746us) server_negotiation.cc:407] Connection header received
0808 18:18:00.837704 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:00.837707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:00.837768 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:00.837879 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:00.839020 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.839615 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.840738 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.840957 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.841069 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:00.843495 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:00.843499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:00.843500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:00.843544 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:00.845122 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:00.845604 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:00.845609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:00.845611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:00.845692 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:00.846062 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:00.846067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:00.846068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:00.846232 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:00.846342 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:00.846769 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 18:18:00.846879 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:18:00.847878 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:00.835843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42473 (local address 127.27.18.194:36717)
0808 18:18:00.836260 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:00.836298 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:00.837505 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:00.838043 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:00.838050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:00.838361 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:00.838876 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:00.838893 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.839740 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.839744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:00.840607 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:00.840616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:00.841108 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:00.841112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:00.841273 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:00.841790 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:00.841806 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:00.843288 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:00.845232 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:00.845238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:00.845250 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:00.845498 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:00.845786 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:00.845801 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:00.845803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:00.845956 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:00.846342 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:00.846348 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:00.846552 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 18:18:00.846724 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 18:18:00.847005 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":106,"threads_started":1}
I20260808 18:18:00.849066 1655 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42473
I20260808 18:18:00.849332 1655 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:00.849790 1655 heartbeater.cc:511] Master 127.27.18.254:42473 requested a full tablet report, sending...
I20260808 18:18:00.850852 1149 ts_manager.cc:194] Registered new tserver with Master: 14236ba9d7894f05bcc72607f64a432e (127.27.18.194:37207)
I20260808 18:18:00.851707 1149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:36717
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:18:00.871097 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42473
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:00.977329 1663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:00.977591 1663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:00.977665 1663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:00.981613 1663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:00.981689 1663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:00.981776 1663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:00.986797 1663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:00.987700 1663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:00.988620 1663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:00.997641 1669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:00.997675 1670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:00.997676 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:00.997810 1663 server_base.cc:1034] running on GCE node
I20260808 18:18:00.998345 1663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:00.998878 1663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:01.000044 1663 hybrid_clock.cc:648] HybridClock initialized: now 1786213081000029 us; error 45 us; skew 500 ppm
Aug 08 18:18:00 dist-test-slave-x4qh krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:01.002581 1663 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:18:01.003497 1663 webserver.cc:460] Webserver started at http://127.27.18.195:36245/ using document root <none> and password file <none>
I20260808 18:18:01.003984 1663 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:01.004087 1663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:01.004315 1663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:01.006264 1663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "f9214e25fd0447e1abb93abd8b81d76f"
format_stamp: "Formatted at 2026-08-08 18:18:01 on dist-test-slave-x4qh"
server_key: "99b266f5db313d05c498851a3e6a064f"
server_key_iv: "35615c199af788d40feba451360a14b3"
server_key_version: "encryptionkey@0"
I20260808 18:18:01.006825 1663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "f9214e25fd0447e1abb93abd8b81d76f"
format_stamp: "Formatted at 2026-08-08 18:18:01 on dist-test-slave-x4qh"
server_key: "99b266f5db313d05c498851a3e6a064f"
server_key_iv: "35615c199af788d40feba451360a14b3"
server_key_version: "encryptionkey@0"
I20260808 18:18:01.010169 1663 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 18:18:01.012374 1684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:01.013811 1663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:18:01.013939 1663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "f9214e25fd0447e1abb93abd8b81d76f"
format_stamp: "Formatted at 2026-08-08 18:18:01 on dist-test-slave-x4qh"
server_key: "99b266f5db313d05c498851a3e6a064f"
server_key_iv: "35615c199af788d40feba451360a14b3"
server_key_version: "encryptionkey@0"
I20260808 18:18:01.014070 1663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:01.045871 1663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:01.048785 1663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:01.048990 1663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:01.049569 1663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:01.050388 1663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:01.050459 1663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:01.050526 1663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:01.050570 1663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:01.059523 1663 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:44033
I20260808 18:18:01.059604 1851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:44033 every 8 connection(s)
I20260808 18:18:01.060436 1663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:01.067113 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 1663
I20260808 18:18:01.067253 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:01.067556 27723 external_mini_cluster.cc:1428] Setting key b3984cdff11b172feeb2af3014402c65
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213080, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:01.074297 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:01.062210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:57133 (local address 127.27.18.254:42473)
0808 18:18:01.062360 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:01.062364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:01.063113 (+ 749us) server_negotiation.cc:407] Connection header received
0808 18:18:01.064571 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:01.064574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:01.064631 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:01.064735 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:01.065812 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.066341 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.067885 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.068195 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.068281 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:01.071079 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:01.071094 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:01.071096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:01.071119 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:01.072670 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:01.073119 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:01.073122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:01.073123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:01.073192 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:01.073481 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:01.073483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:01.073484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:01.073644 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:01.073712 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:01.074103 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 18:18:01.074190 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:18:01.074976 1855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:01.062534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42473 (local address 127.27.18.195:57133)
0808 18:18:01.062969 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:01.063008 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:01.064336 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:01.064906 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:01.064912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:01.065223 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:01.065659 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:01.065671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.066480 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.066491 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:01.067740 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:01.067751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.068346 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.068350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:01.068577 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:01.069211 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:01.069236 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:01.070934 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:01.072774 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:01.072779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:01.072789 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:01.073035 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:01.073270 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:01.073273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:01.073274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:01.073403 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:01.073724 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:01.073728 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:01.073920 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 18:18:01.074061 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 18:18:01.074263 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":125,"threads_started":1}
I20260808 18:18:01.075982 1852 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42473
I20260808 18:18:01.076195 1852 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:01.076612 1852 heartbeater.cc:511] Master 127.27.18.254:42473 requested a full tablet report, sending...
I20260808 18:18:01.077495 1149 ts_manager.cc:194] Registered new tserver with Master: f9214e25fd0447e1abb93abd8b81d76f (127.27.18.195:44033)
I20260808 18:18:01.077957 1149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:57133
I20260808 18:18:01.081616 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:18:01.106155 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:01.097863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.27.18.193:38159)
0808 18:18:01.098177 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:01.098192 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:01.098213 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:18:01.098256 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:01.098260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:01.098397 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:01.098499 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:01.099085 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.099866 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.100589 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.100902 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.100975 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:01.103165 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:01.103173 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:01.103182 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:01.103201 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:01.104662 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:01.105054 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:01.105058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:01.105059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:01.105131 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:01.105350 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:01.105352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:01.105353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:01.105559 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:01.105705 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:01.105833 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 18:18:01.105970 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:01.115490 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:01.108434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.27.18.254:42473)
0808 18:18:01.108547 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:01.108549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:01.108562 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:01.108675 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:01.108676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:01.108709 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:01.108768 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:01.109241 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.109693 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.110352 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:01.110546 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:01.110613 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:01.112723 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:01.112728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:01.112735 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:01.112754 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:01.113967 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:01.114379 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:01.114382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:01.114383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:01.114451 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:01.114678 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:01.114680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:01.114681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:01.114858 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:01.114990 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:01.115191 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 18:18:01.115274 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 18:18:01.115820 1149 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42080
I20260808 18:18:01.116392 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 1248
I20260808 18:18:01.128441 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 1458
I20260808 18:18:01.138571 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 1663
I20260808 18:18:01.153348 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 1091
2026-08-08T18:18:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2213 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1867](info): set up 2 sockets
Aug 08 18:18:01 dist-test-slave-x4qh krb5kdc[1867](info): commencing operation
krb5kdc: starting...
W20260808 18:18:02.338672 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:02 dist-test-slave-x4qh krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:18:02.456238 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44669
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:44669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:02.562074 1892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:02.562288 1892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:02.562327 1892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:02.566110 1892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:02.566154 1892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:02.566172 1892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:02.566188 1892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:02.566202 1892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:02.570824 1892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:44669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:02.571630 1892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:02.572456 1892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:02.580145 1907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:02.580193 1910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:02.580152 1905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:02.580495 1892 server_base.cc:1034] running on GCE node
I20260808 18:18:02.581038 1892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:02.582041 1892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:02.583217 1892 hybrid_clock.cc:648] HybridClock initialized: now 1786213082583193 us; error 38 us; skew 500 ppm
Aug 08 18:18:02 dist-test-slave-x4qh krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213082, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:02.585937 1892 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:18:02.586560 1892 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:02.586616 1892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:02.586756 1892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:02.588603 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "5722c0785bc540b0aebc9e9847f5d56e"
format_stamp: "Formatted at 2026-08-08 18:18:02 on dist-test-slave-x4qh"
server_key: "351b69a9a9d553c19c4d92852288579c"
server_key_iv: "cd8739bfb1e2a94f1dc9fe843f039f60"
server_key_version: "encryptionkey@0"
I20260808 18:18:02.589095 1892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "5722c0785bc540b0aebc9e9847f5d56e"
format_stamp: "Formatted at 2026-08-08 18:18:02 on dist-test-slave-x4qh"
server_key: "351b69a9a9d553c19c4d92852288579c"
server_key_iv: "cd8739bfb1e2a94f1dc9fe843f039f60"
server_key_version: "encryptionkey@0"
I20260808 18:18:02.592227 1892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:02.594182 1921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:02.595331 1892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:18:02.595517 1892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5722c0785bc540b0aebc9e9847f5d56e"
format_stamp: "Formatted at 2026-08-08 18:18:02 on dist-test-slave-x4qh"
server_key: "351b69a9a9d553c19c4d92852288579c"
server_key_iv: "cd8739bfb1e2a94f1dc9fe843f039f60"
server_key_version: "encryptionkey@0"
I20260808 18:18:02.595602 1892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:02.619356 1892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:02.621873 1892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:02.622051 1892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:02.628314 1892 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:44669
I20260808 18:18:02.628590 2005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:44669 every 8 connection(s)
I20260808 18:18:02.628968 1892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:02.631358 2006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:02.632504 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 1892
I20260808 18:18:02.632648 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:02.632965 27723 external_mini_cluster.cc:1428] Setting key 1f31438383ff79ebb667b8af08a27db6
I20260808 18:18:02.637398 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e: Bootstrap starting.
I20260808 18:18:02.639768 2006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:02.640614 2006 log.cc:824] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:18:02 dist-test-slave-x4qh krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:02.642361 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e: No bootstrap required, opened a new log
I20260808 18:18:02.645781 2006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5722c0785bc540b0aebc9e9847f5d56e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44669 } }
I20260808 18:18:02.645998 2006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:02.646073 2006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5722c0785bc540b0aebc9e9847f5d56e, State: Initialized, Role: FOLLOWER
I20260808 18:18:02.646226 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:02.634149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.27.18.254:44669)
0808 18:18:02.634520 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:02.634534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:02.634566 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:18:02.635147 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:02.635161 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:02.635517 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:02.635767 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:02.636593 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:02.637556 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:02.638605 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:02.639006 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:02.639106 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:02.641772 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:02.641777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:02.641788 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:02.641812 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:02.643471 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:02.643878 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:02.643883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:02.643888 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:02.643996 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:02.644281 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:02.644285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:02.644286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:02.644640 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:02.644914 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:02.645216 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 18:18:02.645481 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":134,"threads_started":1}
I20260808 18:18:02.646546 2006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5722c0785bc540b0aebc9e9847f5d56e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44669 } }
I20260808 18:18:02.646684 2006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:02.646745 2006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:02.646831 2006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:02.647836 2006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5722c0785bc540b0aebc9e9847f5d56e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44669 } }
I20260808 18:18:02.648169 2006 leader_election.cc:302] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5722c0785bc540b0aebc9e9847f5d56e; no voters:
I20260808 18:18:02.648510 2006 leader_election.cc:288] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:02.648669 2011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:02.648985 2011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [term 1 LEADER]: Becoming Leader. State: Replica: 5722c0785bc540b0aebc9e9847f5d56e, State: Running, Role: LEADER
I20260808 18:18:02.649310 2011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5722c0785bc540b0aebc9e9847f5d56e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44669 } }
I20260808 18:18:02.649575 2006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:02.650832 2012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5722c0785bc540b0aebc9e9847f5d56e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5722c0785bc540b0aebc9e9847f5d56e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44669 } } }
I20260808 18:18:02.650969 2012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:02.651337 2026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:02.651304 2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5722c0785bc540b0aebc9e9847f5d56e. Latest consensus state: current_term: 1 leader_uuid: "5722c0785bc540b0aebc9e9847f5d56e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5722c0785bc540b0aebc9e9847f5d56e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44669 } } }
I20260808 18:18:02.651429 2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:02.653826 2026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:02.658795 2026 catalog_manager.cc:1347] Generated new cluster ID: dd42fa4824f04cce98963cc94ff3965c
I20260808 18:18:02.658849 2026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:02.680227 2026 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:02.681028 2026 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:02.686872 2026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e: Generated new TSK 0
I20260808 18:18:02.687484 2026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:18:02.719532 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44669
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:02.822783 2039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:02.823026 2039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:02.823122 2039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:02.826784 2039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:02.826867 2039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:02.826967 2039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:02.831979 2039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:02.832921 2039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:02.833844 2039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:02.842512 2046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:02.842525 2053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:02.842597 2039 server_base.cc:1034] running on GCE node
W20260808 18:18:02.842526 2051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:02.843222 2039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:02.843843 2039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:02.845041 2039 hybrid_clock.cc:648] HybridClock initialized: now 1786213082845016 us; error 43 us; skew 500 ppm
Aug 08 18:18:02 dist-test-slave-x4qh krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213082, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:02.847759 2039 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:18:02.848445 2039 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:02.848536 2039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:02.848758 2039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:02.850828 2039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "79b8841375344105bcf4dbd52ecc945f"
format_stamp: "Formatted at 2026-08-08 18:18:02 on dist-test-slave-x4qh"
server_key: "244a23625625fde8094258669fd5bd23"
server_key_iv: "6070388b3c75b328489ae790c314fe40"
server_key_version: "encryptionkey@0"
I20260808 18:18:02.851554 2039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "79b8841375344105bcf4dbd52ecc945f"
format_stamp: "Formatted at 2026-08-08 18:18:02 on dist-test-slave-x4qh"
server_key: "244a23625625fde8094258669fd5bd23"
server_key_iv: "6070388b3c75b328489ae790c314fe40"
server_key_version: "encryptionkey@0"
I20260808 18:18:02.854807 2039 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:18:02.857020 2063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:02.857990 2039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:02.858121 2039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "79b8841375344105bcf4dbd52ecc945f"
format_stamp: "Formatted at 2026-08-08 18:18:02 on dist-test-slave-x4qh"
server_key: "244a23625625fde8094258669fd5bd23"
server_key_iv: "6070388b3c75b328489ae790c314fe40"
server_key_version: "encryptionkey@0"
I20260808 18:18:02.858227 2039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:02.875635 2039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:02.878454 2039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:02.878652 2039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:02.879191 2039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:02.880033 2039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:02.880103 2039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:02.880189 2039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:02.880237 2039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:02.888531 2039 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:44715
I20260808 18:18:02.888571 2237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:44715 every 8 connection(s)
I20260808 18:18:02.889017 2039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:02.894785 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 2039
I20260808 18:18:02.894917 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:02.895202 27723 external_mini_cluster.cc:1428] Setting key 0e6009487c0fd7c22368724cb5ff9709
Aug 08 18:18:02 dist-test-slave-x4qh krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213082, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:02.901435 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:02.890475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:48491 (local address 127.27.18.254:44669)
0808 18:18:02.890621 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:02.890625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:02.891332 (+ 707us) server_negotiation.cc:407] Connection header received
0808 18:18:02.892491 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:02.892494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:02.892538 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:02.892606 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:02.893606 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:02.894087 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:02.895264 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:02.895609 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:02.895722 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:02.898004 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:02.898009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:02.898014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:02.898044 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:02.899422 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:02.900025 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:02.900027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:02.900028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:02.900110 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:02.900487 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:02.900489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:02.900490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:02.900636 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:02.900702 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:02.901245 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 18:18:02.901332 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:18:02.902477 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:02.890797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44669 (local address 127.27.18.193:48491)
0808 18:18:02.891195 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:02.891231 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:02.892319 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:02.892754 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:02.892759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:02.893046 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:02.893492 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:02.893503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:02.894192 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:02.894195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:02.895123 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:02.895139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:02.895756 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:02.895760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:02.895949 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:02.896426 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:02.896443 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:02.897839 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:02.899525 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:02.899533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:02.899548 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:02.899921 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:02.900194 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:02.900199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:02.900204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:02.900375 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:02.900734 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:02.900742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:02.901015 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 18:18:02.901212 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:18:02.901511 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":92,"threads_started":1}
I20260808 18:18:02.903767 2239 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44669
I20260808 18:18:02.904013 2239 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:02.904522 2239 heartbeater.cc:511] Master 127.27.18.254:44669 requested a full tablet report, sending...
I20260808 18:18:02.906212 1947 ts_manager.cc:194] Registered new tserver with Master: 79b8841375344105bcf4dbd52ecc945f (127.27.18.193:44715)
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:18:02.907562 1947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:48491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:18:02.924021 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44669
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:03.029904 2246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:03.030102 2246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:03.030141 2246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:03.033764 2246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:03.033816 2246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:03.033867 2246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:03.038769 2246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:03.039727 2246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:03.040606 2246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:03.049463 2253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:03.049471 2254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:03.049479 2256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.049659 2246 server_base.cc:1034] running on GCE node
I20260808 18:18:03.050148 2246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:03.050774 2246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:03.051930 2246 hybrid_clock.cc:648] HybridClock initialized: now 1786213083051907 us; error 38 us; skew 500 ppm
Aug 08 18:18:03 dist-test-slave-x4qh krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213083, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:03.054544 2246 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:18:03.055194 2246 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:03.055254 2246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:03.055521 2246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:03.057360 2246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "ab41249e279b46108589ac710a58ebc7"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "ec25b3d5adc1dbd5ed78a28a1185c78e"
server_key_iv: "c37b262b750c66007ad01513bf9f9fc1"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.057876 2246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "ab41249e279b46108589ac710a58ebc7"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "ec25b3d5adc1dbd5ed78a28a1185c78e"
server_key_iv: "c37b262b750c66007ad01513bf9f9fc1"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.061233 2246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:03.063360 2266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.064471 2246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:03.064599 2246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "ab41249e279b46108589ac710a58ebc7"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "ec25b3d5adc1dbd5ed78a28a1185c78e"
server_key_iv: "c37b262b750c66007ad01513bf9f9fc1"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.064702 2246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:03.091348 2246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:03.094267 2246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:03.094465 2246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:03.094971 2246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:03.095793 2246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:03.095860 2246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.095922 2246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:03.095979 2246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.104545 2246 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:39013
I20260808 18:18:03.104573 2438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:39013 every 8 connection(s)
I20260808 18:18:03.105046 2246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:03.109604 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 2246
I20260808 18:18:03.109704 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:03.109997 27723 external_mini_cluster.cc:1428] Setting key c60f99ff87ebf1ffc75288a03bafeda4
Aug 08 18:18:03 dist-test-slave-x4qh krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213083, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:03.118403 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.106569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:52971 (local address 127.27.18.254:44669)
0808 18:18:03.106711 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.106714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.107479 (+ 765us) server_negotiation.cc:407] Connection header received
0808 18:18:03.108849 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.108853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.108912 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.109012 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:03.110196 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.110866 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.112172 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.112406 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.112515 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.114845 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:03.114849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:03.114851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:03.114873 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:03.116322 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:03.116934 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:03.116937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:03.116938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:03.117010 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:03.117385 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:03.117387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:03.117388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:03.117533 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:03.117599 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.118200 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.118291 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:18:03.119426 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.106872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44669 (local address 127.27.18.194:52971)
0808 18:18:03.107312 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.107351 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.108630 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.109171 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.109177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.109491 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:03.110005 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.110018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.110992 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.110995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.112042 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.112051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.112564 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.112568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.112732 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.113278 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:03.113293 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:03.114673 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:03.116440 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:03.116457 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:03.116470 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:03.116841 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:03.117102 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:03.117108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:03.117113 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:03.117286 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:03.117642 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:03.117651 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:03.117952 (+ 301us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.118165 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.118487 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":105,"threads_started":1}
I20260808 18:18:03.120471 2439 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44669
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
I20260808 18:18:03.120684 2439 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:03.121162 2439 heartbeater.cc:511] Master 127.27.18.254:44669 requested a full tablet report, sending...
I20260808 18:18:03.122064 1947 ts_manager.cc:194] Registered new tserver with Master: ab41249e279b46108589ac710a58ebc7 (127.27.18.194:39013)
I20260808 18:18:03.122557 1947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:52971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:18:03.136281 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44669
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:03.239991 2451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:03.240234 2451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:03.240314 2451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:03.243865 2451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:03.243932 2451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:03.244025 2451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:03.248831 2451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:03.249660 2451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:03.250572 2451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:03.259189 2459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:03.259168 2463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.259239 2451 server_base.cc:1034] running on GCE node
W20260808 18:18:03.259176 2458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.259689 2451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:03.260267 2451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:03.261445 2451 hybrid_clock.cc:648] HybridClock initialized: now 1786213083261428 us; error 26 us; skew 500 ppm
Aug 08 18:18:03 dist-test-slave-x4qh krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213083, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:03.266607 2451 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:18:03.267261 2451 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:03.267343 2451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:03.267565 2451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:03.269374 2451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "a818bb8f29a44875a1ff2a1987f42fa8"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "16444ddacfaee4fce0f38af0f7e0bf29"
server_key_iv: "041a5c2689bc3e8b8d7de96a672b780c"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.269834 2451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "a818bb8f29a44875a1ff2a1987f42fa8"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "16444ddacfaee4fce0f38af0f7e0bf29"
server_key_iv: "041a5c2689bc3e8b8d7de96a672b780c"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.273197 2451 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:03.275209 2472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.276314 2451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:03.276427 2451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "a818bb8f29a44875a1ff2a1987f42fa8"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "16444ddacfaee4fce0f38af0f7e0bf29"
server_key_iv: "041a5c2689bc3e8b8d7de96a672b780c"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.276515 2451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:03.307431 2451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:03.310024 2451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:03.310220 2451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:03.310765 2451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:03.311510 2451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:03.311554 2451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.311612 2451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:03.311643 2451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.320149 2451 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:44761
I20260808 18:18:03.320161 2656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:44761 every 8 connection(s)
I20260808 18:18:03.320705 2451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:03.321993 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 2451
I20260808 18:18:03.322114 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:03.322427 27723 external_mini_cluster.cc:1428] Setting key 3c6e67f0e584ced6cad9a0daddca9503
Aug 08 18:18:03 dist-test-slave-x4qh krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213083, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:03.333416 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.322409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:50625 (local address 127.27.18.254:44669)
0808 18:18:03.322548 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.322553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.323439 (+ 886us) server_negotiation.cc:407] Connection header received
0808 18:18:03.324961 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.324964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.325010 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.325077 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:03.326224 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.326721 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.327833 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.328052 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.328126 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.330391 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:03.330394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:03.330396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:03.330419 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:03.331829 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:03.332267 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:03.332270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:03.332271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:03.332347 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:03.332616 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:03.332618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:03.332619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:03.332759 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:03.332824 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.333220 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.333302 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:18:03.334147 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.322818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44669 (local address 127.27.18.195:50625)
0808 18:18:03.323280 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.323328 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.324776 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.325240 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.325246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.325572 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:03.326067 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.326086 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.326822 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.326824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.327728 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.327738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.328176 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.328178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.328301 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.328822 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:03.328838 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:03.330223 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:03.331943 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:03.331947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:03.331956 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:03.332188 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:03.332420 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:03.332422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:03.332423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:03.332534 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:03.332835 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:03.332839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:03.333011 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.333167 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.333364 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":137,"threads_started":1}
I20260808 18:18:03.335316 2657 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44669
I20260808 18:18:03.335561 2657 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:03.336066 2657 heartbeater.cc:511] Master 127.27.18.254:44669 requested a full tablet report, sending...
I20260808 18:18:03.337091 1947 ts_manager.cc:194] Registered new tserver with Master: a818bb8f29a44875a1ff2a1987f42fa8 (127.27.18.195:44761)
I20260808 18:18:03.337570 1947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:50625
I20260808 18:18:03.347759 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:18:03.355623 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.349539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46582 (local address 127.27.18.254:44669)
0808 18:18:03.349644 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.349647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.349776 (+ 129us) server_negotiation.cc:407] Connection header received
0808 18:18:03.349888 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.349892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.349936 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.349990 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:03.350585 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.351025 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.351843 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.352037 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.352110 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.353198 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:03.353201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:03.353202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:03.353221 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:03.354320 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:03.354675 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:03.354677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:03.354679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:03.354744 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:03.354967 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:03.354969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:03.354970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:03.355139 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:03.355235 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.355457 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.355534 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:18:03.358080 1947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:03.360889 1947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:03.374053 2680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.368769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:44761 (local address 127.0.0.1:49106)
0808 18:18:03.369161 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.369208 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.369329 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.370038 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.370044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.370065 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:03.370177 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.370197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.371458 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.371461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.372514 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.372525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.373561 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.373565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.373673 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.373708 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.373813 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.373921 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":41,"thread_start_us":136,"threads_started":1}
I20260808 18:18:03.374084 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.368887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.27.18.195:44761)
0808 18:18:03.369309 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.369322 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.369346 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:18:03.369524 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.369530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.369712 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.369856 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:03.370307 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.371290 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.372637 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.373439 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.373539 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.373674 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.373857 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.373946 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":118,"threads_started":1}
I20260808 18:18:03.374886 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.369327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:39013 (local address 127.0.0.1:58840)
0808 18:18:03.369717 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.369730 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.369826 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.370432 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.370435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.370473 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:03.370592 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.370601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.372125 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.372129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.373162 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.373170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.374497 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.374501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.374602 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.374624 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.374699 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.374764 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":22,"thread_start_us":90,"threads_started":1}
I20260808 18:18:03.374989 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.369583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.27.18.194:39013)
0808 18:18:03.369990 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.370004 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.370025 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:18:03.370098 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.370103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.370285 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.370728 (+ 443us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:03.370880 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.371976 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.373300 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.374147 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.374300 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.374455 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.374742 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.374836 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":104,"threads_started":1}
I20260808 18:18:03.375646 2684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.369473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:44715 (local address 127.0.0.1:41556)
0808 18:18:03.370476 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.370502 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.370575 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.371074 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.371077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.371096 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:03.371177 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.371182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.372390 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.372393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.373568 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.373579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.375226 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.375230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.375319 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.375339 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.375454 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.375528 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260808 18:18:03.376171 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.369865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.27.18.193:44715)
0808 18:18:03.370156 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.370163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.370660 (+ 497us) server_negotiation.cc:407] Connection header received
0808 18:18:03.370738 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.370742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.370900 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.371069 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:03.371291 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.372267 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.373704 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.374935 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.375037 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.375159 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.375923 (+ 764us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.376020 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
I20260808 18:18:03.377182 2555 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa4aa08457e4cde88a04800dc1ff1de (DEFAULT_TABLE table=test-table [id=ff1caf0cb19b4c2284028f3d1627edd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:03.377619 2342 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa4aa08457e4cde88a04800dc1ff1de (DEFAULT_TABLE table=test-table [id=ff1caf0cb19b4c2284028f3d1627edd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:03.378289 2555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa4aa08457e4cde88a04800dc1ff1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:03.378567 2342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa4aa08457e4cde88a04800dc1ff1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:03.378597 2133 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa4aa08457e4cde88a04800dc1ff1de (DEFAULT_TABLE table=test-table [id=ff1caf0cb19b4c2284028f3d1627edd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:03.379485 2133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa4aa08457e4cde88a04800dc1ff1de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:03.385668 2690 tablet_bootstrap.cc:492] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Bootstrap starting.
I20260808 18:18:03.387441 2690 tablet_bootstrap.cc:654] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:03.387750 2689 tablet_bootstrap.cc:492] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: Bootstrap starting.
I20260808 18:18:03.388635 2690 log.cc:824] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:03.389238 2691 tablet_bootstrap.cc:492] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Bootstrap starting.
I20260808 18:18:03.389587 2689 tablet_bootstrap.cc:654] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:03.390688 2690 tablet_bootstrap.cc:492] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: No bootstrap required, opened a new log
I20260808 18:18:03.390875 2690 ts_tablet_manager.cc:1397] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:18:03.390870 2689 log.cc:824] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:03.391362 2691 tablet_bootstrap.cc:654] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:03.392381 2691 log.cc:824] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:03.392731 2689 tablet_bootstrap.cc:492] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: No bootstrap required, opened a new log
I20260808 18:18:03.392913 2689 ts_tablet_manager.cc:1397] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:18:03.393878 2691 tablet_bootstrap.cc:492] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: No bootstrap required, opened a new log
I20260808 18:18:03.394063 2691 ts_tablet_manager.cc:1397] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260808 18:18:03.394640 2690 raft_consensus.cc:348] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.394908 2690 raft_consensus.cc:374] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:03.394992 2690 raft_consensus.cc:729] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab41249e279b46108589ac710a58ebc7, State: Initialized, Role: FOLLOWER
I20260808 18:18:03.395466 2690 consensus_queue.cc:260] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.396093 2439 heartbeater.cc:503] Master 127.27.18.254:44669 was elected leader, sending a full tablet report...
I20260808 18:18:03.396375 2690 ts_tablet_manager.cc:1428] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:18:03.397573 2689 raft_consensus.cc:348] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.397810 2689 raft_consensus.cc:374] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:03.397884 2689 raft_consensus.cc:729] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b8841375344105bcf4dbd52ecc945f, State: Initialized, Role: FOLLOWER
I20260808 18:18:03.398293 2689 consensus_queue.cc:260] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.398887 2691 raft_consensus.cc:348] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.399143 2691 raft_consensus.cc:374] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:03.399221 2691 raft_consensus.cc:729] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a818bb8f29a44875a1ff2a1987f42fa8, State: Initialized, Role: FOLLOWER
I20260808 18:18:03.399217 2689 ts_tablet_manager.cc:1428] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:18:03.399614 2239 heartbeater.cc:503] Master 127.27.18.254:44669 was elected leader, sending a full tablet report...
I20260808 18:18:03.399648 2691 consensus_queue.cc:260] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.400218 2657 heartbeater.cc:503] Master 127.27.18.254:44669 was elected leader, sending a full tablet report...
I20260808 18:18:03.400467 2691 ts_tablet_manager.cc:1428] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 18:18:03.474988 2707 raft_consensus.cc:482] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:03.475212 2707 raft_consensus.cc:504] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.476289 2707 leader_election.cc:288] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab41249e279b46108589ac710a58ebc7 (127.27.18.194:39013), 79b8841375344105bcf4dbd52ecc945f (127.27.18.193:44715)
I20260808 18:18:03.479929 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.476596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:39013 (local address 127.27.18.195:44863)
0808 18:18:03.476693 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.476710 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.476817 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.477127 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.477130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.477148 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:03.477276 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.477282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.478006 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.478011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.478778 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.478786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.479445 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.479449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.479612 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.479630 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.479717 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.479789 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":20}
I20260808 18:18:03.479938 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.476704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:44863 (local address 127.27.18.194:39013)
0808 18:18:03.476822 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.476826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.476839 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:03.476973 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.476975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.477014 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.477114 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:03.477399 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.477891 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.478916 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.479298 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.479459 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.479513 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.479745 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.479815 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:18:03.480624 2370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa4aa08457e4cde88a04800dc1ff1de" candidate_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab41249e279b46108589ac710a58ebc7" is_pre_election: true
I20260808 18:18:03.480840 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.476596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:44715 (local address 127.27.18.195:60215)
0808 18:18:03.477327 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.477339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:03.477457 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:03.477714 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:03.477717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:03.477733 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:03.477860 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.477865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.478624 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.478628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.479609 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:03.479618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.480509 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.480512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:03.480589 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.480606 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:18:03.480662 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 18:18:03.480702 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":419,"thread_start_us":148,"threads_started":1}
I20260808 18:18:03.480890 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.476677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:60215 (local address 127.27.18.193:44715)
0808 18:18:03.476819 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.476825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.477425 (+ 600us) server_negotiation.cc:407] Connection header received
0808 18:18:03.477571 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.477575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.477614 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.477692 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:03.477971 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.478516 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.479726 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.480346 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.480500 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.480541 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.480709 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.480783 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:18:03.480944 2370 raft_consensus.cc:2454] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818bb8f29a44875a1ff2a1987f42fa8 in term 0.
I20260808 18:18:03.481458 2474 leader_election.cc:302] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a818bb8f29a44875a1ff2a1987f42fa8, ab41249e279b46108589ac710a58ebc7; no voters:
I20260808 18:18:03.481508 2166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa4aa08457e4cde88a04800dc1ff1de" candidate_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b8841375344105bcf4dbd52ecc945f" is_pre_election: true
I20260808 18:18:03.481727 2707 raft_consensus.cc:2781] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:03.481801 2707 raft_consensus.cc:482] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:03.481770 2166 raft_consensus.cc:2454] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a818bb8f29a44875a1ff2a1987f42fa8 in term 0.
I20260808 18:18:03.481859 2707 raft_consensus.cc:3037] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:03.482868 2707 raft_consensus.cc:504] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.483198 2707 leader_election.cc:288] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [CANDIDATE]: Term 1 election: Requested vote from peers ab41249e279b46108589ac710a58ebc7 (127.27.18.194:39013), 79b8841375344105bcf4dbd52ecc945f (127.27.18.193:44715)
I20260808 18:18:03.483537 2370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa4aa08457e4cde88a04800dc1ff1de" candidate_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab41249e279b46108589ac710a58ebc7"
I20260808 18:18:03.483567 2166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa4aa08457e4cde88a04800dc1ff1de" candidate_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b8841375344105bcf4dbd52ecc945f"
I20260808 18:18:03.483664 2370 raft_consensus.cc:3037] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:03.483664 2166 raft_consensus.cc:3037] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:03.484964 2166 raft_consensus.cc:2454] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818bb8f29a44875a1ff2a1987f42fa8 in term 1.
I20260808 18:18:03.484964 2370 raft_consensus.cc:2454] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a818bb8f29a44875a1ff2a1987f42fa8 in term 1.
I20260808 18:18:03.485247 2474 leader_election.cc:302] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a818bb8f29a44875a1ff2a1987f42fa8, ab41249e279b46108589ac710a58ebc7; no voters:
I20260808 18:18:03.485446 2707 raft_consensus.cc:2781] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:03.485623 2707 raft_consensus.cc:686] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 1 LEADER]: Becoming Leader. State: Replica: a818bb8f29a44875a1ff2a1987f42fa8, State: Running, Role: LEADER
I20260808 18:18:03.485924 2707 consensus_queue.cc:237] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } }
I20260808 18:18:03.488760 1946 catalog_manager.cc:5495] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a818bb8f29a44875a1ff2a1987f42fa8 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a818bb8f29a44875a1ff2a1987f42fa8" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 44761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20260808 18:18:03.523319 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.520362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.27.18.195:44761)
0808 18:18:03.520483 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.520487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.520509 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:18:03.520599 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.520601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.520647 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.520730 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:03.520938 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.521404 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.522180 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:03.522380 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:03.522448 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:03.522674 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:03.523040 (+ 366us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:03.523127 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.523178 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.523218 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:03.528906 2166 raft_consensus.cc:1261] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f [term 1 FOLLOWER]: Refusing update from remote peer a818bb8f29a44875a1ff2a1987f42fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:03.528906 2370 raft_consensus.cc:1261] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 1 FOLLOWER]: Refusing update from remote peer a818bb8f29a44875a1ff2a1987f42fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:03.529623 2707 consensus_queue.cc:1035] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b8841375344105bcf4dbd52ecc945f" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:03.529844 2710 consensus_queue.cc:1035] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab41249e279b46108589ac710a58ebc7" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:03.536849 2717 mvcc.cc:205] Tried to move back new op lower bound from 7316328790127312896 to 7316328789960880128. Current Snapshot: MvccSnapshot[applied={T|T < 7316328790127312896}]
I20260808 18:18:03.543891 1947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:03.544071 1947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:03.546470 1947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5722c0785bc540b0aebc9e9847f5d56e: Sending DeleteTablet for 3 replicas of tablet 2aa4aa08457e4cde88a04800dc1ff1de
I20260808 18:18:03.547128 2555 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa4aa08457e4cde88a04800dc1ff1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:03 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:49106
I20260808 18:18:03.547179 2133 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa4aa08457e4cde88a04800dc1ff1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:03 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:41556
I20260808 18:18:03.547613 2342 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa4aa08457e4cde88a04800dc1ff1de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:03 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:58840
I20260808 18:18:03.547878 2733 tablet_replica.cc:330] T 2aa4aa08457e4cde88a04800dc1ff1de P 79b8841375344105bcf4dbd52ecc945f: stopping tablet replica
I20260808 18:18:03.547688 2732 tablet_replica.cc:330] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: stopping tablet replica
I20260808 18:18:03.548020 2734 tablet_replica.cc:330] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: stopping tablet replica
I20260808 18:18:03.548153 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 2039
I20260808 18:18:03.551091 2732 raft_consensus.cc:2229] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:03.551517 2732 raft_consensus.cc:2258] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:03.551558 2734 raft_consensus.cc:2229] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:03.551853 2734 raft_consensus.cc:2258] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:03.553119 2732 ts_tablet_manager.cc:1905] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:03.553297 2734 ts_tablet_manager.cc:1905] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:03.555943 2732 ts_tablet_manager.cc:1918] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:03.556095 2732 log.cc:1196] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/2aa4aa08457e4cde88a04800dc1ff1de
I20260808 18:18:03.556541 2732 ts_tablet_manager.cc:1939] T 2aa4aa08457e4cde88a04800dc1ff1de P a818bb8f29a44875a1ff2a1987f42fa8: Deleting consensus metadata
I20260808 18:18:03.556659 2734 ts_tablet_manager.cc:1918] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:03.556771 2734 log.cc:1196] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/2aa4aa08457e4cde88a04800dc1ff1de
I20260808 18:18:03.557123 2734 ts_tablet_manager.cc:1939] T 2aa4aa08457e4cde88a04800dc1ff1de P ab41249e279b46108589ac710a58ebc7: Deleting consensus metadata
I20260808 18:18:03.557782 1927 catalog_manager.cc:4841] TS a818bb8f29a44875a1ff2a1987f42fa8 (127.27.18.195:44761): tablet 2aa4aa08457e4cde88a04800dc1ff1de (table test-table [id=ff1caf0cb19b4c2284028f3d1627edd2]) successfully deleted
I20260808 18:18:03.558224 1925 catalog_manager.cc:4841] TS ab41249e279b46108589ac710a58ebc7 (127.27.18.194:39013): tablet 2aa4aa08457e4cde88a04800dc1ff1de (table test-table [id=ff1caf0cb19b4c2284028f3d1627edd2]) successfully deleted
W20260808 18:18:03.560535 1927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:44715 (error 108)
I20260808 18:18:03.560674 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 2246
I20260808 18:18:03.563591 2684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.563230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:44715 (local address 127.0.0.1:41572)
0808 18:18:03.563456 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:03.563516 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:44715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260808 18:18:03.563742 1927 catalog_manager.cc:4584] TS 79b8841375344105bcf4dbd52ecc945f (127.27.18.193:44715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aa4aa08457e4cde88a04800dc1ff1de: Network error: Client connection negotiation failed: client connection to 127.27.18.193:44715: connect: Connection refused (error 111)
I20260808 18:18:03.572032 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 2451
I20260808 18:18:03.587185 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 1892
2026-08-08T18:18:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2431 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T18:18:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:03Z Disabled control of system clock
I20260808 18:18:03.618750 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33445
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:33445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:18:03.721665 2744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:03.721874 2744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:03.721925 2744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:03.725824 2744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:03.725870 2744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:03.725886 2744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:03.725903 2744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:03.725935 2744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:03.730245 2744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:33445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:03.731074 2744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:03.732287 2744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:03.740317 2752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.740383 2744 server_base.cc:1034] running on GCE node
W20260808 18:18:03.740319 2753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:03.740342 2755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.741158 2744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:03.742074 2744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:03.743249 2744 hybrid_clock.cc:648] HybridClock initialized: now 1786213083743225 us; error 32 us; skew 500 ppm
I20260808 18:18:03.745173 2744 webserver.cc:460] Webserver started at http://127.27.18.254:41167/ using document root <none> and password file <none>
I20260808 18:18:03.745726 2744 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:03.745790 2744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:03.746037 2744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:03.748023 2744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "996c6a5bda16464a8fa66f8602ca64d3"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "f86a5ba15ed9195c419043c8df277239"
server_key_iv: "9d20e439391cad0a005714e55d74db9a"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.748574 2744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "996c6a5bda16464a8fa66f8602ca64d3"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "f86a5ba15ed9195c419043c8df277239"
server_key_iv: "9d20e439391cad0a005714e55d74db9a"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.752064 2744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:03.754483 2761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.755673 2744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:03.755806 2744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "996c6a5bda16464a8fa66f8602ca64d3"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "f86a5ba15ed9195c419043c8df277239"
server_key_iv: "9d20e439391cad0a005714e55d74db9a"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.755913 2744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:03.760357 2744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:03.760914 2744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:03.761098 2744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:03.768193 2744 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:33445
I20260808 18:18:03.768199 2833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:33445 every 8 connection(s)
I20260808 18:18:03.769035 2744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:03.772266 2834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:03.773850 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 2744
I20260808 18:18:03.773944 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:03.774246 27723 external_mini_cluster.cc:1428] Setting key d240718b74f333766bba69e2f50d5813
I20260808 18:18:03.779028 2834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: Bootstrap starting.
I20260808 18:18:03.780133 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:03.775661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.27.18.254:33445)
0808 18:18:03.776328 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:03.776349 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:03.776391 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:18:03.777151 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:03.777177 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:03.777603 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:03.777972 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:03.778691 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:03.778697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:03.778711 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:03.778723 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:03.778787 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:03.778809 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:03.778948 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:03.779013 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 18:18:03.779291 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":108,"threads_started":1}
I20260808 18:18:03.781587 2834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:03.782326 2834 log.cc:824] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:03.784163 2834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: No bootstrap required, opened a new log
I20260808 18:18:03.787631 2834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "996c6a5bda16464a8fa66f8602ca64d3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33445 } }
I20260808 18:18:03.787859 2834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:03.787936 2834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996c6a5bda16464a8fa66f8602ca64d3, State: Initialized, Role: FOLLOWER
I20260808 18:18:03.788375 2834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "996c6a5bda16464a8fa66f8602ca64d3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33445 } }
I20260808 18:18:03.788525 2834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:03.788625 2834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:03.788741 2834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:03.789556 2834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "996c6a5bda16464a8fa66f8602ca64d3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33445 } }
I20260808 18:18:03.789885 2834 leader_election.cc:302] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 996c6a5bda16464a8fa66f8602ca64d3; no voters:
I20260808 18:18:03.790174 2834 leader_election.cc:288] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:03.790263 2840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:03.790542 2840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [term 1 LEADER]: Becoming Leader. State: Replica: 996c6a5bda16464a8fa66f8602ca64d3, State: Running, Role: LEADER
I20260808 18:18:03.790829 2840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "996c6a5bda16464a8fa66f8602ca64d3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33445 } }
I20260808 18:18:03.791471 2834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:03.792516 2844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 996c6a5bda16464a8fa66f8602ca64d3. Latest consensus state: current_term: 1 leader_uuid: "996c6a5bda16464a8fa66f8602ca64d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "996c6a5bda16464a8fa66f8602ca64d3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33445 } } }
I20260808 18:18:03.792670 2844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:03.792516 2841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "996c6a5bda16464a8fa66f8602ca64d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "996c6a5bda16464a8fa66f8602ca64d3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33445 } } }
I20260808 18:18:03.792783 2841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3 [sys.catalog]: This master's current role is: LEADER
W20260808 18:18:03.796427 2869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:18:03.796525 2869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:18:03.796604 2871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:03.797266 2871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:03.802827 2871 catalog_manager.cc:1347] Generated new cluster ID: f1bb54f06a79436588625ad54b4fec33
I20260808 18:18:03.802992 2871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:03.813437 2871 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:03.813740 2871 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:03.819589 2871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: Generated new TSK 0
I20260808 18:18:03.820179 2871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:18:03.827343 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33445
--builtin_ntp_servers=127.27.18.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:18:03.964560 2876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:03.964792 2876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:03.964835 2876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:03.968614 2876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:03.968657 2876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:03.968740 2876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:03.973121 2876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:03.974040 2876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:03.975332 2876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:03.984833 2881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:03.984807 2884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.984961 2876 server_base.cc:1034] running on GCE node
W20260808 18:18:03.985066 2887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:03.985523 2876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:03.986093 2876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:03.987272 2876 hybrid_clock.cc:648] HybridClock initialized: now 1786213083987250 us; error 57 us; skew 500 ppm
I20260808 18:18:03.988996 2876 webserver.cc:460] Webserver started at http://127.27.18.193:39629/ using document root <none> and password file <none>
I20260808 18:18:03.989508 2876 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:03.989614 2876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:03.989841 2876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:03.991812 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "97f4377d645b4c938f86707f48193618"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "d6e59cde4dbfa9072db41db050192e20"
server_key_iv: "83aa97ef085bbeb0c7009c8440de3cd6"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.992345 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "97f4377d645b4c938f86707f48193618"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "d6e59cde4dbfa9072db41db050192e20"
server_key_iv: "83aa97ef085bbeb0c7009c8440de3cd6"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.995787 2876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:03.997889 2898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:03.998935 2876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:03.999068 2876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "97f4377d645b4c938f86707f48193618"
format_stamp: "Formatted at 2026-08-08 18:18:03 on dist-test-slave-x4qh"
server_key: "d6e59cde4dbfa9072db41db050192e20"
server_key_iv: "83aa97ef085bbeb0c7009c8440de3cd6"
server_key_version: "encryptionkey@0"
I20260808 18:18:03.999171 2876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:04.003149 2876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:04.003782 2876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:04.003960 2876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:04.004519 2876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:04.005353 2876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:04.005422 2876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.005481 2876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:04.005542 2876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.014370 2876 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:37873
I20260808 18:18:04.014417 3080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:37873 every 8 connection(s)
I20260808 18:18:04.015146 2876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:04.020757 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.016725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:57641 (local address 127.27.18.254:33445)
0808 18:18:04.016863 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.016868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.017752 (+ 884us) server_negotiation.cc:407] Connection header received
0808 18:18:04.019073 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.019077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.019123 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.019199 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.020297 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.020301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.020304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.020316 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.020337 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.020346 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.020447 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.020587 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.020630 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:18:04.021483 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.017043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33445 (local address 127.27.18.193:57641)
0808 18:18:04.017619 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.017654 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.018866 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.019353 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.019359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.019494 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.020054 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.020055 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.020090 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.020150 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.020436 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.020445 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.020452 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.020626 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.020836 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
I20260808 18:18:04.022574 3082 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33445
I20260808 18:18:04.022753 3082 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:04.023068 3082 heartbeater.cc:511] Master 127.27.18.254:33445 requested a full tablet report, sending...
I20260808 18:18:04.024712 2788 ts_manager.cc:194] Registered new tserver with Master: 97f4377d645b4c938f86707f48193618 (127.27.18.193:37873)
I20260808 18:18:04.025408 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 2876
I20260808 18:18:04.025530 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:04.025846 27723 external_mini_cluster.cc:1428] Setting key fccfb6f46795832d079e379a7a33040a
I20260808 18:18:04.027624 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33445
--builtin_ntp_servers=127.27.18.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:18:04.139189 3089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:04.139449 3089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:04.139519 3089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:04.144606 3089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:04.144654 3089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:04.144744 3089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:04.150012 3089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:04.150906 3089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:04.152194 3089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:04.161657 3102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:04.161656 3098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:04.161696 3100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:04.161949 3089 server_base.cc:1034] running on GCE node
I20260808 18:18:04.162398 3089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:04.162998 3089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:04.164179 3089 hybrid_clock.cc:648] HybridClock initialized: now 1786213084164170 us; error 30 us; skew 500 ppm
I20260808 18:18:04.165884 3089 webserver.cc:460] Webserver started at http://127.27.18.194:41695/ using document root <none> and password file <none>
I20260808 18:18:04.166412 3089 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:04.166500 3089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:04.166692 3089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:04.168637 3089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "365c655c988047598e75a58cd022d4f1"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "1f19f989b3b52ca1e9bb2cbc138cfe37"
server_key_iv: "92e673a76dcaff1b002bcfb58cfed42a"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.169164 3089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "365c655c988047598e75a58cd022d4f1"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "1f19f989b3b52ca1e9bb2cbc138cfe37"
server_key_iv: "92e673a76dcaff1b002bcfb58cfed42a"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.172904 3089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:04.175474 3115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.176641 3089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:04.176784 3089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "365c655c988047598e75a58cd022d4f1"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "1f19f989b3b52ca1e9bb2cbc138cfe37"
server_key_iv: "92e673a76dcaff1b002bcfb58cfed42a"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.176930 3089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:04.181653 3089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:04.182245 3089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:04.182440 3089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:04.182984 3089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:04.183990 3089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:04.184067 3089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.184158 3089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:04.184209 3089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.193987 3089 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:35565
I20260808 18:18:04.194056 3281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:35565 every 8 connection(s)
I20260808 18:18:04.194851 3089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:04.200583 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.196783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:56595 (local address 127.27.18.254:33445)
0808 18:18:04.196927 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.196930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.197889 (+ 959us) server_negotiation.cc:407] Connection header received
0808 18:18:04.199120 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.199122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.199158 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.199229 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.200230 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.200232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.200234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.200246 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.200264 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.200271 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.200337 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.200451 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.200487 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:18:04.201340 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.197105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33445 (local address 127.27.18.194:56595)
0808 18:18:04.197702 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.197774 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.198947 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.199482 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.199489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.199574 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.200012 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.200014 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.200047 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.200104 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.200341 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.200347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.200351 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.200467 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.200625 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":119,"threads_started":1}
I20260808 18:18:04.202308 3282 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33445
I20260808 18:18:04.202507 3282 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:04.202872 3282 heartbeater.cc:511] Master 127.27.18.254:33445 requested a full tablet report, sending...
I20260808 18:18:04.203812 2788 ts_manager.cc:194] Registered new tserver with Master: 365c655c988047598e75a58cd022d4f1 (127.27.18.194:35565)
I20260808 18:18:04.204079 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 3089
I20260808 18:18:04.204213 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:04.204496 27723 external_mini_cluster.cc:1428] Setting key 3533d3a3999f068bc391069639a6d41d
I20260808 18:18:04.206372 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33445
--builtin_ntp_servers=127.27.18.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 18:18:04.313185 3288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:04.313432 3288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:04.313470 3288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:04.317315 3288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:04.317359 3288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:04.317409 3288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:04.322021 3288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:04.322939 3288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:04.324221 3288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:04.333544 3296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:04.333614 3299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:04.333525 3297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:04.333793 3288 server_base.cc:1034] running on GCE node
I20260808 18:18:04.334218 3288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:04.334877 3288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:04.336030 3288 hybrid_clock.cc:648] HybridClock initialized: now 1786213084336005 us; error 36 us; skew 500 ppm
I20260808 18:18:04.337836 3288 webserver.cc:460] Webserver started at http://127.27.18.195:37503/ using document root <none> and password file <none>
I20260808 18:18:04.338380 3288 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:04.338442 3288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:04.338701 3288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:04.340698 3288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "ced4e63e1d114e238a57cef7e6af46f2"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "8b8831a1c3c53aa049fe2c15882bca45"
server_key_iv: "2dc38feed99ed4495819aa9332e2c2a0"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.341235 3288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "ced4e63e1d114e238a57cef7e6af46f2"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "8b8831a1c3c53aa049fe2c15882bca45"
server_key_iv: "2dc38feed99ed4495819aa9332e2c2a0"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.344774 3288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:18:04.347206 3308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.348366 3288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:04.348508 3288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "ced4e63e1d114e238a57cef7e6af46f2"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "8b8831a1c3c53aa049fe2c15882bca45"
server_key_iv: "2dc38feed99ed4495819aa9332e2c2a0"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.348618 3288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:04.352917 3288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:04.353456 3288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:04.353634 3288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:04.354166 3288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:04.355006 3288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:04.355075 3288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.355140 3288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:04.355187 3288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.364998 3288 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46215
I20260808 18:18:04.365041 3481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46215 every 8 connection(s)
I20260808 18:18:04.365826 3288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:04.371506 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.367574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:52425 (local address 127.27.18.254:33445)
0808 18:18:04.367732 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.367737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.368784 (+ 1047us) server_negotiation.cc:407] Connection header received
0808 18:18:04.370014 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.370017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.370052 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.370124 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.371078 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.371081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.371082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.371093 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.371112 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.371121 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.371192 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.371311 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.371347 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:04.372257 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.368001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33445 (local address 127.27.18.195:52425)
0808 18:18:04.368650 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.368687 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.369827 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.370276 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.370282 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.370360 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.370841 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.370843 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.370880 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.370942 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.371194 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.371200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.371204 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.371324 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.371527 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260808 18:18:04.372974 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 3288
I20260808 18:18:04.373101 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:04.373415 27723 external_mini_cluster.cc:1428] Setting key a1a21b8be9ef108a63d4063fa201e06f
I20260808 18:18:04.373531 3482 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33445
I20260808 18:18:04.373766 3482 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:04.374238 3482 heartbeater.cc:511] Master 127.27.18.254:33445 requested a full tablet report, sending...
I20260808 18:18:04.376183 2788 ts_manager.cc:194] Registered new tserver with Master: ced4e63e1d114e238a57cef7e6af46f2 (127.27.18.195:46215)
I20260808 18:18:04.387069 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:18:04.390599 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.389028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60288 (local address 127.27.18.254:33445)
0808 18:18:04.389144 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.389146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.389347 (+ 201us) server_negotiation.cc:407] Connection header received
0808 18:18:04.389437 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.389439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.389492 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.389557 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.390320 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.390322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.390324 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.390335 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.390350 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.390358 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.390422 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.390484 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.390512 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:18:04.393154 2788 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:04.396129 2788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:04.407547 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.404512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46215 (local address 127.0.0.1:34732)
0808 18:18:04.405300 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.405358 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.405453 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.406032 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.406038 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.406105 (+ 67us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.406650 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.406653 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.406672 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.406735 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.407210 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.407221 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.407227 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.407331 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.407403 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":84,"threads_started":1}
I20260808 18:18:04.407886 3500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.404405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:37873 (local address 127.0.0.1:38792)
0808 18:18:04.405056 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.405506 (+ 450us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.405569 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.406775 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.406778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.406792 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.407226 (+ 434us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.407227 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.407237 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.407262 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.407626 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.407629 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.407632 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.407702 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.407736 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":85,"threads_started":1}
I20260808 18:18:04.408139 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.404546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.27.18.193:37873)
0808 18:18:04.405834 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.405851 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.405871 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:18:04.405914 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.405917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.406097 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.406231 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.407441 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.407448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.407458 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.407470 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.407508 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.407536 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.407768 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.407820 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.407896 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174,"thread_start_us":127,"threads_started":1}
I20260808 18:18:04.408201 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.405192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.27.18.195:46215)
0808 18:18:04.405560 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.405576 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.405602 (+ 26us) server_negotiation.cc:407] Connection header received
0808 18:18:04.405657 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.405664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.405868 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.406018 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.406910 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.406915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.406924 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.406934 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.407011 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.407041 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.407163 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.407984 (+ 821us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.408045 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":139,"threads_started":1}
I20260808 18:18:04.408224 3501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.404763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:35565 (local address 127.0.0.1:43460)
0808 18:18:04.405652 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.405666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.405732 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.407021 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.407024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.407036 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.407412 (+ 376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.407415 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.407426 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.407452 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.407917 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.407923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.407929 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.408012 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.408046 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":94,"threads_started":1}
I20260808 18:18:04.408653 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.405181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.27.18.194:35565)
0808 18:18:04.406642 (+ 1461us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.406653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.406675 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:18:04.406729 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.406736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.406892 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.407034 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.407696 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.407701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.407712 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.407722 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.407755 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.407778 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.407872 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.408426 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.408491 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1363,"thread_start_us":199,"threads_started":1}
I20260808 18:18:04.410967 3380 tablet_service.cc:1467] Processing CreateTablet for tablet ba24d36e5fe94591a30063ac7e964ace (DEFAULT_TABLE table=test-table [id=90614334eba24d0593b101eee3a01368]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:04.411296 3186 tablet_service.cc:1467] Processing CreateTablet for tablet ba24d36e5fe94591a30063ac7e964ace (DEFAULT_TABLE table=test-table [id=90614334eba24d0593b101eee3a01368]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:04.411352 2976 tablet_service.cc:1467] Processing CreateTablet for tablet ba24d36e5fe94591a30063ac7e964ace (DEFAULT_TABLE table=test-table [id=90614334eba24d0593b101eee3a01368]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:04.411984 3380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba24d36e5fe94591a30063ac7e964ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:04.412361 2976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba24d36e5fe94591a30063ac7e964ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:04.412371 3186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba24d36e5fe94591a30063ac7e964ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:04.419749 3510 tablet_bootstrap.cc:492] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Bootstrap starting.
I20260808 18:18:04.419894 3512 tablet_bootstrap.cc:492] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: Bootstrap starting.
I20260808 18:18:04.421931 3512 tablet_bootstrap.cc:654] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:04.422173 3510 tablet_bootstrap.cc:654] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:04.422425 3516 tablet_bootstrap.cc:492] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Bootstrap starting.
I20260808 18:18:04.422825 3512 log.cc:824] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:04.423015 3510 log.cc:824] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:04.424587 3516 tablet_bootstrap.cc:654] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:04.424937 3512 tablet_bootstrap.cc:492] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: No bootstrap required, opened a new log
I20260808 18:18:04.424962 3510 tablet_bootstrap.cc:492] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: No bootstrap required, opened a new log
I20260808 18:18:04.425127 3512 ts_tablet_manager.cc:1397] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:18:04.425151 3510 ts_tablet_manager.cc:1397] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 18:18:04.425415 3516 log.cc:824] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:04.427281 3516 tablet_bootstrap.cc:492] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: No bootstrap required, opened a new log
I20260808 18:18:04.427510 3516 ts_tablet_manager.cc:1397] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:18:04.428601 3510 raft_consensus.cc:348] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.428880 3510 raft_consensus.cc:374] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:04.428966 3510 raft_consensus.cc:729] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ced4e63e1d114e238a57cef7e6af46f2, State: Initialized, Role: FOLLOWER
I20260808 18:18:04.429306 3512 raft_consensus.cc:348] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.429533 3512 raft_consensus.cc:374] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:04.429435 3510 consensus_queue.cc:260] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.429628 3512 raft_consensus.cc:729] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f4377d645b4c938f86707f48193618, State: Initialized, Role: FOLLOWER
I20260808 18:18:04.430044 3482 heartbeater.cc:503] Master 127.27.18.254:33445 was elected leader, sending a full tablet report...
I20260808 18:18:04.430053 3512 consensus_queue.cc:260] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.431064 3512 ts_tablet_manager.cc:1428] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:18:04.431334 3510 ts_tablet_manager.cc:1428] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260808 18:18:04.431452 3082 heartbeater.cc:503] Master 127.27.18.254:33445 was elected leader, sending a full tablet report...
I20260808 18:18:04.432030 3516 raft_consensus.cc:348] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.432248 3516 raft_consensus.cc:374] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:04.432319 3516 raft_consensus.cc:729] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365c655c988047598e75a58cd022d4f1, State: Initialized, Role: FOLLOWER
I20260808 18:18:04.432749 3516 consensus_queue.cc:260] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.433382 3282 heartbeater.cc:503] Master 127.27.18.254:33445 was elected leader, sending a full tablet report...
I20260808 18:18:04.433653 3516 ts_tablet_manager.cc:1428] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 18:18:04.445799 3284 tablet.cc:2369] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:04.456342 3523 raft_consensus.cc:482] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:04.456498 3523 raft_consensus.cc:504] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.457418 3523 leader_election.cc:288] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ced4e63e1d114e238a57cef7e6af46f2 (127.27.18.195:46215), 365c655c988047598e75a58cd022d4f1 (127.27.18.194:35565)
I20260808 18:18:04.459407 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.457661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46215 (local address 127.27.18.193:41037)
0808 18:18:04.457797 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.457815 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.457898 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.458167 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.458170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.458185 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.458701 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.458705 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.458718 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.458749 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.459079 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.459085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.459091 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.459183 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.459217 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 18:18:04.459425 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.457753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:41037 (local address 127.27.18.195:46215)
0808 18:18:04.457884 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.457888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.457904 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:18:04.457984 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.457990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.458031 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.458149 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.458917 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.458921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.458924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.458938 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.458960 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.458973 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.459064 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.459190 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.459231 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:18:04.459667 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.457661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:35565 (local address 127.27.18.193:41709)
0808 18:18:04.458231 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.458244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.458338 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.458572 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.458575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.458589 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:04.459006 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:04.459007 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:04.459015 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:04.459058 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:04.459337 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:04.459339 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:04.459342 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:18:04.459495 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 18:18:04.459531 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":235,"thread_start_us":117,"threads_started":1}
I20260808 18:18:04.459728 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.457850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:41709 (local address 127.27.18.194:35565)
0808 18:18:04.457989 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.457993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.458352 (+ 359us) server_negotiation.cc:407] Connection header received
0808 18:18:04.458414 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.458418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.458451 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.458523 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.459171 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.459175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.459178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.459190 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.459211 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.459220 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.459332 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.459515 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.459562 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:18:04.460269 3413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba24d36e5fe94591a30063ac7e964ace" candidate_uuid: "97f4377d645b4c938f86707f48193618" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced4e63e1d114e238a57cef7e6af46f2" is_pre_election: true
I20260808 18:18:04.460390 3207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba24d36e5fe94591a30063ac7e964ace" candidate_uuid: "97f4377d645b4c938f86707f48193618" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "365c655c988047598e75a58cd022d4f1" is_pre_election: true
I20260808 18:18:04.460556 3413 raft_consensus.cc:2454] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97f4377d645b4c938f86707f48193618 in term 0.
I20260808 18:18:04.460608 3207 raft_consensus.cc:2454] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97f4377d645b4c938f86707f48193618 in term 0.
I20260808 18:18:04.460996 2904 leader_election.cc:302] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365c655c988047598e75a58cd022d4f1, 97f4377d645b4c938f86707f48193618; no voters:
I20260808 18:18:04.461264 3523 raft_consensus.cc:2781] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:04.461345 3523 raft_consensus.cc:482] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:04.461376 3523 raft_consensus.cc:3037] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 0 FOLLOWER]: Advancing to term 1
W20260808 18:18:04.516196 3085 tablet.cc:2369] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:04.522855 3523 raft_consensus.cc:504] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.523396 3523 leader_election.cc:288] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [CANDIDATE]: Term 1 election: Requested vote from peers ced4e63e1d114e238a57cef7e6af46f2 (127.27.18.195:46215), 365c655c988047598e75a58cd022d4f1 (127.27.18.194:35565)
I20260808 18:18:04.523873 3413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba24d36e5fe94591a30063ac7e964ace" candidate_uuid: "97f4377d645b4c938f86707f48193618" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced4e63e1d114e238a57cef7e6af46f2"
I20260808 18:18:04.523936 3207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba24d36e5fe94591a30063ac7e964ace" candidate_uuid: "97f4377d645b4c938f86707f48193618" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "365c655c988047598e75a58cd022d4f1"
I20260808 18:18:04.524061 3207 raft_consensus.cc:3037] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:04.524042 3413 raft_consensus.cc:3037] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:04.525560 3413 raft_consensus.cc:2454] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97f4377d645b4c938f86707f48193618 in term 1.
I20260808 18:18:04.525560 3207 raft_consensus.cc:2454] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97f4377d645b4c938f86707f48193618 in term 1.
I20260808 18:18:04.525962 2904 leader_election.cc:302] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365c655c988047598e75a58cd022d4f1, 97f4377d645b4c938f86707f48193618; no voters:
I20260808 18:18:04.526183 3523 raft_consensus.cc:2781] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:04.526397 3523 raft_consensus.cc:686] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 1 LEADER]: Becoming Leader. State: Replica: 97f4377d645b4c938f86707f48193618, State: Running, Role: LEADER
I20260808 18:18:04.526777 3523 consensus_queue.cc:237] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } }
I20260808 18:18:04.530475 2788 catalog_manager.cc:5495] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97f4377d645b4c938f86707f48193618 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "97f4377d645b4c938f86707f48193618" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f4377d645b4c938f86707f48193618" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 } health_report { overall_health: UNKNOWN } } }
I20260808 18:18:04.561497 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.559930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.27.18.193:37873)
0808 18:18:04.560054 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.560058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.560072 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:04.560273 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.560275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.560347 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.560425 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.561181 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.561184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.561186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.561194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.561209 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.561220 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.561302 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.561360 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.561394 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:18:04.567284 3207 raft_consensus.cc:1261] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 1 FOLLOWER]: Refusing update from remote peer 97f4377d645b4c938f86707f48193618: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:04.567284 3413 raft_consensus.cc:1261] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 1 FOLLOWER]: Refusing update from remote peer 97f4377d645b4c938f86707f48193618: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:04.568028 3523 consensus_queue.cc:1035] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ced4e63e1d114e238a57cef7e6af46f2" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:04.568336 3526 consensus_queue.cc:1035] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [LEADER]: Connected to new peer: Peer: permanent_uuid: "365c655c988047598e75a58cd022d4f1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:04.577339 3534 mvcc.cc:205] Tried to move back new op lower bound from 7316328794380689408 to 7316328794225041408. Current Snapshot: MvccSnapshot[applied={T|T < 7316328794380689408}]
I20260808 18:18:04.583757 2788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:04.583918 2788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:04.586247 2788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 996c6a5bda16464a8fa66f8602ca64d3: Sending DeleteTablet for 3 replicas of tablet ba24d36e5fe94591a30063ac7e964ace
I20260808 18:18:04.586947 2976 tablet_service.cc:1514] Processing DeleteTablet for tablet ba24d36e5fe94591a30063ac7e964ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:04 UTC) from {username='slave'} at 127.0.0.1:38792
I20260808 18:18:04.586997 3380 tablet_service.cc:1514] Processing DeleteTablet for tablet ba24d36e5fe94591a30063ac7e964ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:04 UTC) from {username='slave'} at 127.0.0.1:34732
I20260808 18:18:04.587502 3547 tablet_replica.cc:330] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618: stopping tablet replica
I20260808 18:18:04.587517 3186 tablet_service.cc:1514] Processing DeleteTablet for tablet ba24d36e5fe94591a30063ac7e964ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:04 UTC) from {username='slave'} at 127.0.0.1:43460
I20260808 18:18:04.587841 3547 raft_consensus.cc:2229] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:04.588050 3549 tablet_replica.cc:330] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: stopping tablet replica
I20260808 18:18:04.588176 3547 raft_consensus.cc:2258] T ba24d36e5fe94591a30063ac7e964ace P 97f4377d645b4c938f86707f48193618 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:04.588536 3546 tablet_replica.cc:330] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: stopping tablet replica
I20260808 18:18:04.588559 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 2876
I20260808 18:18:04.588824 3546 raft_consensus.cc:2229] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:04.589087 3546 raft_consensus.cc:2258] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:04.594112 3549 raft_consensus.cc:2229] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:04.595142 3546 ts_tablet_manager.cc:1905] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:04.595564 3549 raft_consensus.cc:2258] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:04.597960 3549 ts_tablet_manager.cc:1905] T ba24d36e5fe94591a30063ac7e964ace P 365c655c988047598e75a58cd022d4f1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:18:04.603734 2763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:37873 (error 108)
I20260808 18:18:04.603881 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 3089
I20260808 18:18:04.606482 3501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.606043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:37873 (local address 127.0.0.1:38800)
0808 18:18:04.606278 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.606365 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260808 18:18:04.606706 2763 catalog_manager.cc:4584] TS 97f4377d645b4c938f86707f48193618 (127.27.18.193:37873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba24d36e5fe94591a30063ac7e964ace: Network error: Client connection negotiation failed: client connection to 127.27.18.193:37873: connect: Connection refused (error 111)
I20260808 18:18:04.629333 3546 ts_tablet_manager.cc:1918] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:04.629493 3546 log.cc:1196] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/ba24d36e5fe94591a30063ac7e964ace
I20260808 18:18:04.629842 3546 ts_tablet_manager.cc:1939] T ba24d36e5fe94591a30063ac7e964ace P ced4e63e1d114e238a57cef7e6af46f2: Deleting consensus metadata
I20260808 18:18:04.634254 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 3288
I20260808 18:18:04.636986 3501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.634449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:35565 (local address 127.0.0.1:43464)
0808 18:18:04.636807 (+ 2358us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.636876 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.194:35565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2246}
I20260808 18:18:04.651176 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 2744
2026-08-08T18:18:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1064 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 18:18:04.675283 27723 mini_oidc.cc:152] Starting JWKS server
I20260808 18:18:04.676988 27723 webserver.cc:460] Webserver started at https://127.0.0.1:43055/ using document root <none> and password file <none>
I20260808 18:18:04.677165 27723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:18:04.677479 27723 webserver.cc:460] Webserver started at http://127.0.0.1:35237/ using document root <none> and password file <none>
2026-08-08T18:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:04Z Disabled control of system clock
I20260808 18:18:04.686419 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36663
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:38043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:36663
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43055/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/testchainca.cert with env {}
W20260808 18:18:04.790695 3559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:04.790899 3559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:04.790933 3559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:04.794632 3559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:04.794679 3559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:04.794694 3559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:04.794749 3559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:04.794777 3559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:18:04.794817 3559 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:18:04.794848 3559 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43055/jwks.json
I20260808 18:18:04.799608 3559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:36663
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43055/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:04.800441 3559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:04.801262 3559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:04.809010 3573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:04.809074 3565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:04.809093 3566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:04.809144 3559 server_base.cc:1034] running on GCE node
I20260808 18:18:04.809780 3559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:04.810822 3559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:04.812016 3559 hybrid_clock.cc:648] HybridClock initialized: now 1786213084812001 us; error 38 us; skew 500 ppm
I20260808 18:18:04.813858 3559 webserver.cc:460] Webserver started at http://127.27.18.254:43873/ using document root <none> and password file <none>
I20260808 18:18:04.814414 3559 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:04.814507 3559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:04.814731 3559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:04.816697 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "93a03b7d454445ff955823de3008fae4"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "be732b8f7d4a2290a81cc3bb6e5063e0"
server_key_iv: "3da53c8073105d9a127f454fb4be7092"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.817235 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "93a03b7d454445ff955823de3008fae4"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "be732b8f7d4a2290a81cc3bb6e5063e0"
server_key_iv: "3da53c8073105d9a127f454fb4be7092"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.820529 3559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:04.822611 3581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:04.823721 3559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:04.823866 3559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "93a03b7d454445ff955823de3008fae4"
format_stamp: "Formatted at 2026-08-08 18:18:04 on dist-test-slave-x4qh"
server_key: "be732b8f7d4a2290a81cc3bb6e5063e0"
server_key_iv: "3da53c8073105d9a127f454fb4be7092"
server_key_version: "encryptionkey@0"
I20260808 18:18:04.823971 3559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:04.843643 3559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:04.844182 3559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:04.844376 3559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:04.850898 3559 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:36663
I20260808 18:18:04.850907 3671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:36663 every 8 connection(s)
I20260808 18:18:04.851713 3559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:04.854411 3672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:04.859177 3672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4: Bootstrap starting.
I20260808 18:18:04.861399 3672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:04.861850 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 3559
I20260808 18:18:04.861977 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:04.862162 3672 log.cc:824] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:04.862248 27723 external_mini_cluster.cc:1428] Setting key 945901a5576008ba8236e991447a49ca
I20260808 18:18:04.864136 3672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4: No bootstrap required, opened a new log
I20260808 18:18:04.867201 3672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a03b7d454445ff955823de3008fae4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36663 } }
I20260808 18:18:04.867455 3672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:04.867542 3672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93a03b7d454445ff955823de3008fae4, State: Initialized, Role: FOLLOWER
I20260808 18:18:04.867978 3672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a03b7d454445ff955823de3008fae4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36663 } }
I20260808 18:18:04.868136 3672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:04.868211 3672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:04.868314 3672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:04.869174 3672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a03b7d454445ff955823de3008fae4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36663 } }
I20260808 18:18:04.869529 3672 leader_election.cc:302] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93a03b7d454445ff955823de3008fae4; no voters:
I20260808 18:18:04.869851 3672 leader_election.cc:288] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:04.869946 3681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:04.870146 3681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [term 1 LEADER]: Becoming Leader. State: Replica: 93a03b7d454445ff955823de3008fae4, State: Running, Role: LEADER
I20260808 18:18:04.870443 3681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a03b7d454445ff955823de3008fae4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36663 } }
I20260808 18:18:04.870736 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.863249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52660 (local address 127.27.18.254:36663)
0808 18:18:04.863848 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.863862 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.863892 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:18:04.864532 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.864549 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.864830 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.865118 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:04.865526 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:04.866368 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:04.867202 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:04.867818 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:04.867995 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:04.868844 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:04.868850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:04.868869 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:04.869198 (+ 329us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:04.869245 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:04.869262 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:04.869384 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:04.869528 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 18:18:04.869787 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":153,"threads_started":1}
I20260808 18:18:04.870997 3672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:04.872103 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93a03b7d454445ff955823de3008fae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a03b7d454445ff955823de3008fae4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36663 } } }
I20260808 18:18:04.872222 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:04.872570 3684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93a03b7d454445ff955823de3008fae4. Latest consensus state: current_term: 1 leader_uuid: "93a03b7d454445ff955823de3008fae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93a03b7d454445ff955823de3008fae4" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36663 } } }
I20260808 18:18:04.872663 3684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:04.872931 3696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:04.873703 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:18:04.876197 3696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:04.880999 3696 catalog_manager.cc:1347] Generated new cluster ID: 2d70d8395ae44247835d762ba9799f9e
I20260808 18:18:04.881064 3696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:04.890573 3696 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:04.891328 3696 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:04.896886 3696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93a03b7d454445ff955823de3008fae4: Generated new TSK 0
I20260808 18:18:04.897420 3696 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 18:18:04.930305 3720 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 18:18:04.930475 3720 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 18:18:04.930938 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:04.914020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.27.18.254:36663)
0808 18:18:04.914169 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:04.914174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:04.914409 (+ 235us) server_negotiation.cc:407] Connection header received
0808 18:18:04.914638 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:04.914640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:04.914691 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:04.914768 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:18:04.915079 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:04.915711 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:04.916927 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:04.917137 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:04.917216 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:04.917607 (+ 391us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:18:04.930651 (+ 13044us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43055/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 18:18:04.930845 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43055/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:18:04.931032 3680 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43055/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 18:18:04.932552 3719 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:04.913961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36663 (local address 127.0.0.1:52674)
0808 18:18:04.914320 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:04.914335 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:04.914479 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:04.914814 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:04.914817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:04.914825 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:18:04.914965 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:04.914972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:04.915837 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:04.915841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:04.916789 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:04.916799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:04.917270 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:04.917276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:04.917371 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:04.917423 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:18:04.930845 (+ 13422us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43055/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 18:18:04.931052 (+ 207us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.18.254:36663: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43055/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":121,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.27.18.254:36663: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43055/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 18:18:04.933218 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 3559
2026-08-08T18:18:04Z chronyd exiting
I20260808 18:18:04.982905 27723 test_util.cc:175] -----------------------------------------------
I20260808 18:18:04.982975 27723 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786213058405439-27723-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 18:18:04.984067 27723 mini_oidc.cc:152] Starting JWKS server
I20260808 18:18:04.984880 27723 webserver.cc:460] Webserver started at https://127.0.0.1:33265/ using document root <none> and password file <none>
I20260808 18:18:04.984994 27723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:18:04.985283 27723 webserver.cc:460] Webserver started at http://127.0.0.1:40479/ using document root <none> and password file <none>
2026-08-08T18:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:04Z Disabled control of system clock
I20260808 18:18:04.994275 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:40315
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:36817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:40315
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33265/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/testchainca.cert with env {}
W20260808 18:18:05.097523 3727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:05.097741 3727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:05.097775 3727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:05.101353 3727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:05.101397 3727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:05.101411 3727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:05.101439 3727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:05.101452 3727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:18:05.101485 3727 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:18:05.101497 3727 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33265/jwks.json
I20260808 18:18:05.106247 3727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:36817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:40315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:40315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33265/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:05.107066 3727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:05.107923 3727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:05.116025 3735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:05.116036 3739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:05.116083 3734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:05.116066 3727 server_base.cc:1034] running on GCE node
I20260808 18:18:05.116688 3727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:05.117594 3727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:05.118743 3727 hybrid_clock.cc:648] HybridClock initialized: now 1786213085118717 us; error 39 us; skew 500 ppm
I20260808 18:18:05.120479 3727 webserver.cc:460] Webserver started at http://127.27.18.254:38533/ using document root <none> and password file <none>
I20260808 18:18:05.120957 3727 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:05.121012 3727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:05.121157 3727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:05.122912 3727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "4c64e3354fd6461788cd9b4714f659cc"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "4b566a075a906b5a8f70b4ac69eece18"
server_key_iv: "a98fdaf34c9d1a02e3ccc9424ef42370"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.123430 3727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "4c64e3354fd6461788cd9b4714f659cc"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "4b566a075a906b5a8f70b4ac69eece18"
server_key_iv: "a98fdaf34c9d1a02e3ccc9424ef42370"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.126616 3727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:18:05.128687 3748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:05.129791 3727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:05.129891 3727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4c64e3354fd6461788cd9b4714f659cc"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "4b566a075a906b5a8f70b4ac69eece18"
server_key_iv: "a98fdaf34c9d1a02e3ccc9424ef42370"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.129967 3727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:05.175061 3727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:05.175611 3727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:05.175743 3727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:05.181985 3727 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:40315
I20260808 18:18:05.181989 3843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:40315 every 8 connection(s)
I20260808 18:18:05.182778 3727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:05.185835 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:05.189558 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 3727
I20260808 18:18:05.189671 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:05.189955 27723 external_mini_cluster.cc:1428] Setting key 617c402d70ba4170a55a9e8643c4e432
I20260808 18:18:05.191200 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc: Bootstrap starting.
I20260808 18:18:05.193179 3844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:05.193830 3844 log.cc:824] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:05.195525 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc: No bootstrap required, opened a new log
I20260808 18:18:05.198241 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:05.191130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.27.18.254:40315)
0808 18:18:05.191737 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:05.191751 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:05.191783 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:18:05.192341 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:05.192355 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:05.192666 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:05.192950 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:05.193387 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.194290 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.195181 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.195714 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.195866 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.196673 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:05.196685 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:05.196696 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:05.197052 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:05.197105 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:05.197124 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:05.197230 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:05.197403 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 18:18:05.197636 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":131,"threads_started":1}
I20260808 18:18:05.199276 3844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c64e3354fd6461788cd9b4714f659cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40315 } }
I20260808 18:18:05.199529 3844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:05.199626 3844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c64e3354fd6461788cd9b4714f659cc, State: Initialized, Role: FOLLOWER
I20260808 18:18:05.200078 3844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c64e3354fd6461788cd9b4714f659cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40315 } }
I20260808 18:18:05.200238 3844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:05.200307 3844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:05.200398 3844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:05.200733 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:18:05.201330 3844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c64e3354fd6461788cd9b4714f659cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40315 } }
I20260808 18:18:05.201673 3844 leader_election.cc:302] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c64e3354fd6461788cd9b4714f659cc; no voters:
I20260808 18:18:05.201968 3844 leader_election.cc:288] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:05.202071 3855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:05.202283 3855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [term 1 LEADER]: Becoming Leader. State: Replica: 4c64e3354fd6461788cd9b4714f659cc, State: Running, Role: LEADER
I20260808 18:18:05.202565 3855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c64e3354fd6461788cd9b4714f659cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40315 } }
I20260808 18:18:05.203055 3844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:05.204116 3857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c64e3354fd6461788cd9b4714f659cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c64e3354fd6461788cd9b4714f659cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40315 } } }
I20260808 18:18:05.204250 3857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:05.204507 3856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c64e3354fd6461788cd9b4714f659cc. Latest consensus state: current_term: 1 leader_uuid: "4c64e3354fd6461788cd9b4714f659cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c64e3354fd6461788cd9b4714f659cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 40315 } } }
I20260808 18:18:05.204655 3856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:05.204861 3867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:05.207321 3867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:05.212028 3867 catalog_manager.cc:1347] Generated new cluster ID: 5e52b6e4daf54913bed217d387dcac7b
I20260808 18:18:05.212085 3867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:05.231913 3867 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:05.232959 3867 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:05.248944 3867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c64e3354fd6461788cd9b4714f659cc: Generated new TSK 0
I20260808 18:18:05.249691 3867 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 18:18:05.272313 3890 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 18:18:05.272399 3890 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 18:18:05.273005 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:05.257373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.27.18.254:40315)
0808 18:18:05.257501 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:05.257506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:05.257656 (+ 150us) server_negotiation.cc:407] Connection header received
0808 18:18:05.257763 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:05.257766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:05.257818 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:05.257944 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:18:05.258157 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.259018 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.260180 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.260574 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.260690 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.260970 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:18:05.272639 (+ 11669us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 18:18:05.272881 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 18:18:05.273065 3853 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 18:18:05.273042 3889 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:05.257256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:40315 (local address 127.0.0.1:57830)
0808 18:18:05.257564 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:05.257575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:05.257668 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:05.257949 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:05.257951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:05.257957 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:18:05.258054 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:05.258058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.259155 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.259182 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:05.260064 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:05.260074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.260705 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.260708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:05.260785 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.260806 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:18:05.272791 (+ 11985us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 18:18:05.272919 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.18.254:40315: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":115,"threads_started":1}
I20260808 18:18:05.273734 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 3727
2026-08-08T18:18:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 18:18:05.322206 27723 mini_oidc.cc:152] Starting JWKS server
I20260808 18:18:05.322989 27723 webserver.cc:460] Webserver started at https://127.0.0.1:40367/ using document root <none> and password file <none>
I20260808 18:18:05.323064 27723 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 18:18:05.323253 27723 webserver.cc:460] Webserver started at http://127.0.0.1:45907/ using document root <none> and password file <none>
2026-08-08T18:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:05Z Disabled control of system clock
I20260808 18:18:05.332773 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:45295
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:45295
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40367/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 18:18:05.433964 3898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:05.434173 3898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:05.434206 3898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:05.437849 3898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:05.437904 3898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:05.437918 3898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:05.437932 3898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:05.437947 3898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:18:05.437960 3898 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:18:05.437975 3898 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40367/jwks.json
I20260808 18:18:05.442400 3898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:45295
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:45295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40367/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:05.443216 3898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:05.444466 3898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:05.452575 3909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:05.452574 3910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:05.452626 3898 server_base.cc:1034] running on GCE node
W20260808 18:18:05.452591 3913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:05.453310 3898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:05.454257 3898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:05.455442 3898 hybrid_clock.cc:648] HybridClock initialized: now 1786213085455427 us; error 35 us; skew 500 ppm
I20260808 18:18:05.457301 3898 webserver.cc:460] Webserver started at http://127.27.18.254:44629/ using document root <none> and password file <none>
I20260808 18:18:05.457845 3898 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:05.457939 3898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:05.458177 3898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:05.460260 3898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "cde4d956776549179623c46d1eda1f8a"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "1917e6998455380529e3ad44e2abe3b1"
server_key_iv: "9842211ecbbca74ed52ad61d88b3b9e3"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.460816 3898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "cde4d956776549179623c46d1eda1f8a"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "1917e6998455380529e3ad44e2abe3b1"
server_key_iv: "9842211ecbbca74ed52ad61d88b3b9e3"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.464223 3898 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:05.466542 3921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:05.467730 3898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:05.467869 3898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "cde4d956776549179623c46d1eda1f8a"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "1917e6998455380529e3ad44e2abe3b1"
server_key_iv: "9842211ecbbca74ed52ad61d88b3b9e3"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.467996 3898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:05.488667 3898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:05.489284 3898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:05.489478 3898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:05.496091 3898 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:45295
I20260808 18:18:05.496098 4007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:45295 every 8 connection(s)
I20260808 18:18:05.496955 3898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:05.498296 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 3898
I20260808 18:18:05.498435 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:05.498734 27723 external_mini_cluster.cc:1428] Setting key 333dccb3ae7f122f03c9876ec881c99b
I20260808 18:18:05.500353 4008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:05.506621 4008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a: Bootstrap starting.
I20260808 18:18:05.507649 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:05.499913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.27.18.254:45295)
0808 18:18:05.500521 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:05.500536 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:05.500570 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:18:05.501145 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:05.501158 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:05.501518 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:05.501756 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:05.502122 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.503049 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.504203 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.504644 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.504754 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.505557 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:05.505562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:05.505572 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:05.506076 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:05.506145 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:05.506176 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:05.506276 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:05.506473 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 18:18:05.506749 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":135,"threads_started":1}
I20260808 18:18:05.509547 4008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:05.510466 4008 log.cc:824] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:05.512411 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:18:05.512624 4008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a: No bootstrap required, opened a new log
I20260808 18:18:05.516076 4008 raft_consensus.cc:348] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde4d956776549179623c46d1eda1f8a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45295 } }
I20260808 18:18:05.516291 4008 raft_consensus.cc:374] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:05.516371 4008 raft_consensus.cc:729] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde4d956776549179623c46d1eda1f8a, State: Initialized, Role: FOLLOWER
I20260808 18:18:05.516755 4008 consensus_queue.cc:260] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde4d956776549179623c46d1eda1f8a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45295 } }
I20260808 18:18:05.516897 4008 raft_consensus.cc:388] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:05.516984 4008 raft_consensus.cc:482] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:05.517100 4008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:05.517894 4008 raft_consensus.cc:504] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde4d956776549179623c46d1eda1f8a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45295 } }
I20260808 18:18:05.518242 4008 leader_election.cc:302] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cde4d956776549179623c46d1eda1f8a; no voters:
I20260808 18:18:05.518541 4008 leader_election.cc:288] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:05.518625 4017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:05.518895 4017 raft_consensus.cc:686] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [term 1 LEADER]: Becoming Leader. State: Replica: cde4d956776549179623c46d1eda1f8a, State: Running, Role: LEADER
I20260808 18:18:05.519172 4017 consensus_queue.cc:237] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde4d956776549179623c46d1eda1f8a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45295 } }
I20260808 18:18:05.519763 4008 sys_catalog.cc:564] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:05.520819 4019 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cde4d956776549179623c46d1eda1f8a. Latest consensus state: current_term: 1 leader_uuid: "cde4d956776549179623c46d1eda1f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde4d956776549179623c46d1eda1f8a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45295 } } }
I20260808 18:18:05.520978 4019 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:05.521385 4028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:05.521941 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cde4d956776549179623c46d1eda1f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cde4d956776549179623c46d1eda1f8a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45295 } } }
I20260808 18:18:05.522038 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:05.523991 4028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:05.529124 4028 catalog_manager.cc:1347] Generated new cluster ID: 08fe68efef7f4b84a81a82fd864bbae5
I20260808 18:18:05.529183 4028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:05.539602 4028 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:05.540338 4028 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:05.548506 4028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cde4d956776549179623c46d1eda1f8a: Generated new TSK 0
I20260808 18:18:05.549077 4028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:18:05.568395 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:05.563579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.27.18.254:45295)
0808 18:18:05.563709 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:05.563714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:05.563887 (+ 173us) server_negotiation.cc:407] Connection header received
0808 18:18:05.564020 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:05.564021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:05.564062 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:05.564141 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:18:05.564389 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.564872 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.566147 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.566346 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.566420 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.566725 (+ 305us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:18:05.568072 (+ 1347us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40367/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 18:18:05.568277 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40367/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 18:18:05.568451 4013 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40367/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 18:18:05.570255 4052 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:05.563507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45295 (local address 127.0.0.1:50800)
0808 18:18:05.563789 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:05.563814 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:05.563921 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:05.564166 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:05.564169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:05.564177 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:18:05.564287 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:05.564292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.564979 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.564983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:05.566034 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:05.566044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.566458 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.566462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:05.566549 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.566576 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:18:05.568230 (+ 1654us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40367/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 18:18:05.570094 (+ 1864us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.18.254:45295: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40367/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":1676,"thread_start_us":110,"threads_started":1}
I20260808 18:18:05.570721 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 3898
E20260808 18:18:05.585433 4055 webserver.cc:533] Webserver: sslize failed:
E20260808 18:18:05.585500 4055 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T18:18:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (295 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T18:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:05Z Disabled control of system clock
I20260808 18:18:05.626703 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:39847
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:39847
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/testchainca.cert with env {}
W20260808 18:18:05.728186 4062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:05.728389 4062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:05.728421 4062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:05.731911 4062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:05.731950 4062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:05.731964 4062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:05.731980 4062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:05.731994 4062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 18:18:05.732009 4062 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 18:18:05.732023 4062 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 18:18:05.736804 4062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:39847
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:39847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:05.737536 4062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:05.738296 4062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:05.745620 4069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:05.745643 4073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:05.745669 4068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:05.745808 4062 server_base.cc:1034] running on GCE node
I20260808 18:18:05.746424 4062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:05.747359 4062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:05.748574 4062 hybrid_clock.cc:648] HybridClock initialized: now 1786213085748559 us; error 29 us; skew 500 ppm
I20260808 18:18:05.750108 4062 webserver.cc:460] Webserver started at http://127.27.18.254:36093/ using document root <none> and password file <none>
I20260808 18:18:05.750592 4062 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:05.750679 4062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:05.750870 4062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:05.752710 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "ce7a196f89d34360b90117b03e46f802"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "4c4e5469dccda55523ffc5334f85f5da"
server_key_iv: "bae6e61c3150b6a2187c1fba36355d83"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.753233 4062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "ce7a196f89d34360b90117b03e46f802"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "4c4e5469dccda55523ffc5334f85f5da"
server_key_iv: "bae6e61c3150b6a2187c1fba36355d83"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.756347 4062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 18:18:05.758502 4082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:05.759516 4062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:05.759639 4062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "ce7a196f89d34360b90117b03e46f802"
format_stamp: "Formatted at 2026-08-08 18:18:05 on dist-test-slave-x4qh"
server_key: "4c4e5469dccda55523ffc5334f85f5da"
server_key_iv: "bae6e61c3150b6a2187c1fba36355d83"
server_key_version: "encryptionkey@0"
I20260808 18:18:05.759737 4062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:05.796340 4062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:05.796980 4062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:05.797173 4062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:05.803544 4062 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:39847
I20260808 18:18:05.803555 4155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:39847 every 8 connection(s)
I20260808 18:18:05.804389 4062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:05.806885 4156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:05.811930 4156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802: Bootstrap starting.
I20260808 18:18:05.812373 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 4062
I20260808 18:18:05.812497 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:05.812809 27723 external_mini_cluster.cc:1428] Setting key 66647e43f6e78f7f09d5ef1965afdff0
I20260808 18:18:05.814538 4156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:05.815212 4156 log.cc:824] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:05.816985 4156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802: No bootstrap required, opened a new log
I20260808 18:18:05.820222 4156 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7a196f89d34360b90117b03e46f802" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 39847 } }
I20260808 18:18:05.820466 4156 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:05.820530 4156 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce7a196f89d34360b90117b03e46f802, State: Initialized, Role: FOLLOWER
I20260808 18:18:05.820935 4156 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7a196f89d34360b90117b03e46f802" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 39847 } }
I20260808 18:18:05.821076 4156 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:05.821156 4156 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:05.821255 4156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:05.821704 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:05.813922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.27.18.254:39847)
0808 18:18:05.814509 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:05.814522 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:05.814556 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:18:05.815185 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:05.815201 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:05.815650 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:05.815993 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:05.816381 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.817408 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.818376 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.818847 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.819022 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.819906 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:05.819911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:05.819922 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:05.820252 (+ 330us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:05.820302 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:05.820323 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:05.820439 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:05.820589 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 18:18:05.820831 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":120,"threads_started":1}
I20260808 18:18:05.822227 4156 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7a196f89d34360b90117b03e46f802" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 39847 } }
I20260808 18:18:05.822554 4156 leader_election.cc:302] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce7a196f89d34360b90117b03e46f802; no voters:
I20260808 18:18:05.822875 4156 leader_election.cc:288] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:05.822954 4163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:05.823155 4163 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [term 1 LEADER]: Becoming Leader. State: Replica: ce7a196f89d34360b90117b03e46f802, State: Running, Role: LEADER
I20260808 18:18:05.823721 4163 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7a196f89d34360b90117b03e46f802" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 39847 } }
I20260808 18:18:05.824076 4156 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:05.824429 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:18:05.825268 4164 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce7a196f89d34360b90117b03e46f802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7a196f89d34360b90117b03e46f802" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 39847 } } }
I20260808 18:18:05.825390 4164 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:05.825728 4166 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce7a196f89d34360b90117b03e46f802. Latest consensus state: current_term: 1 leader_uuid: "ce7a196f89d34360b90117b03e46f802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7a196f89d34360b90117b03e46f802" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 39847 } } }
I20260808 18:18:05.825855 4166 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:05.826119 4179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:05.828483 4179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:05.833572 4179 catalog_manager.cc:1347] Generated new cluster ID: 332e2e299a7c4fd095646b68fb80a72e
I20260808 18:18:05.833640 4179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:05.863699 4179 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:05.864547 4179 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:05.872040 4179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce7a196f89d34360b90117b03e46f802: Generated new TSK 0
I20260808 18:18:05.872568 4179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:18:06.325500 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:05.884858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.27.18.254:39847)
0808 18:18:05.884973 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:05.884979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:05.885123 (+ 144us) server_negotiation.cc:407] Connection header received
0808 18:18:05.885241 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:05.885244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:05.885294 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:05.885402 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0808 18:18:05.885602 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.886357 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.887238 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.887633 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.887736 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.887998 (+ 262us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 18:18:06.325123 (+437125us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 18:18:06.325382 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37546: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 18:18:06.325575 4160 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37546: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 18:18:06.325819 4198 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:05.884757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:39847 (local address 127.0.0.1:37546)
0808 18:18:05.885029 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:05.885041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:05.885129 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:05.885407 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:05.885409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:05.885414 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0808 18:18:05.885498 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:05.885502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.886485 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.886488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:05.887118 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:05.887125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:05.887748 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:05.887751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:05.887821 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:05.887841 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 18:18:06.325369 (+437528us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 18:18:06.325538 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.18.254:39847: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
I20260808 18:18:06.326378 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 4062
2026-08-08T18:18:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (733 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (121 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:06 dist-test-slave-x4qh krb5kdc[4215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:06 dist-test-slave-x4qh krb5kdc[4215](info): set up 2 sockets
Aug 08 18:18:06 dist-test-slave-x4qh krb5kdc[4215](info): commencing operation
krb5kdc: starting...
W20260808 18:18:07.763706 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:07 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:18:07.880016 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42265
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:07.983891 4232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:07.984097 4232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:07.984130 4232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:07.987779 4232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:07.987819 4232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:07.987838 4232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:07.987854 4232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:07.987869 4232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:07.992339 4232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:07.993121 4232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:07.993942 4232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:08.001828 4242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.001842 4243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.001875 4249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:08.001895 4232 server_base.cc:1034] running on GCE node
I20260808 18:18:08.002508 4232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:08.003537 4232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:08.004738 4232 hybrid_clock.cc:648] HybridClock initialized: now 1786213088004717 us; error 32 us; skew 500 ppm
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.007457 4232 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:18:08.008275 4232 webserver.cc:460] Webserver started at http://127.27.18.254:35663/ using document root <none> and password file <none>
I20260808 18:18:08.008759 4232 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:08.008850 4232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:08.009063 4232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:08.010771 4232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "79bd1b704ac64b1c9f3d744b669da5b8"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "0bc18f87267002879977c2fbc742e539"
server_key_iv: "1cff360a4003b6195d646386937ca273"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.011278 4232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "79bd1b704ac64b1c9f3d744b669da5b8"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "0bc18f87267002879977c2fbc742e539"
server_key_iv: "1cff360a4003b6195d646386937ca273"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.015189 4232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:18:08.017578 4262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.018697 4232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:08.018837 4232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "79bd1b704ac64b1c9f3d744b669da5b8"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "0bc18f87267002879977c2fbc742e539"
server_key_iv: "1cff360a4003b6195d646386937ca273"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.018942 4232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:08.030716 4232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:08.033622 4232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:08.033792 4232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:08.041051 4232 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:42265
I20260808 18:18:08.041061 4342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:42265 every 8 connection(s)
I20260808 18:18:08.042026 4232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:08.044556 4343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:08.045655 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 4232
I20260808 18:18:08.045816 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:08.046118 27723 external_mini_cluster.cc:1428] Setting key 21eba5ad0c5a28adb35de8d1ed68cf13
I20260808 18:18:08.050153 4343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8: Bootstrap starting.
I20260808 18:18:08.052343 4343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:08.053150 4343 log.cc:824] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:08.055272 4343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8: No bootstrap required, opened a new log
I20260808 18:18:08.059119 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.047466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.27.18.254:42265)
0808 18:18:08.047875 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.047888 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.047918 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:18:08.048445 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.048459 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.048711 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.048996 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.049771 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.050719 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.051646 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.052129 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.052290 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.054948 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.054953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.054965 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.054989 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.056570 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.057066 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.057070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.057074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.057166 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.057475 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.057476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.057477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.057780 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.057920 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.058172 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.058434 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":137,"threads_started":1}
I20260808 18:18:08.059098 4343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42265 } }
I20260808 18:18:08.059355 4343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:08.059442 4343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bd1b704ac64b1c9f3d744b669da5b8, State: Initialized, Role: FOLLOWER
I20260808 18:18:08.059883 4343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42265 } }
I20260808 18:18:08.059996 4343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:08.060075 4343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:08.060177 4343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:08.061120 4343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42265 } }
I20260808 18:18:08.061503 4343 leader_election.cc:302] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79bd1b704ac64b1c9f3d744b669da5b8; no voters:
I20260808 18:18:08.061802 4343 leader_election.cc:288] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:08.061915 4350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:08.062165 4350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [term 1 LEADER]: Becoming Leader. State: Replica: 79bd1b704ac64b1c9f3d744b669da5b8, State: Running, Role: LEADER
I20260808 18:18:08.062466 4350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42265 } }
I20260808 18:18:08.062834 4343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:08.064116 4354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79bd1b704ac64b1c9f3d744b669da5b8. Latest consensus state: current_term: 1 leader_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42265 } } }
I20260808 18:18:08.064143 4351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79bd1b704ac64b1c9f3d744b669da5b8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42265 } } }
I20260808 18:18:08.064250 4354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:08.064250 4351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:08.065018 4364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:08.068392 4364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:08.073021 4364 catalog_manager.cc:1347] Generated new cluster ID: a01d30be650d434d89caa2eafe9dbd0b
I20260808 18:18:08.073124 4364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:08.080859 4364 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:08.081547 4364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:08.104557 4364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79bd1b704ac64b1c9f3d744b669da5b8: Generated new TSK 0
I20260808 18:18:08.105108 4364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:18:08.140563 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42265
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:08.261284 4381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:08.261502 4381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:08.261554 4381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:08.265429 4381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:08.265478 4381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:08.265578 4381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:08.270262 4381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:08.271109 4381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:08.272073 4381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:08.280783 4389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.280826 4390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.280848 4392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:08.280980 4381 server_base.cc:1034] running on GCE node
I20260808 18:18:08.281291 4381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:08.281850 4381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:08.282999 4381 hybrid_clock.cc:648] HybridClock initialized: now 1786213088282977 us; error 38 us; skew 500 ppm
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.288729 4381 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:18:08.289454 4381 webserver.cc:460] Webserver started at http://127.27.18.193:38825/ using document root <none> and password file <none>
I20260808 18:18:08.289896 4381 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:08.289947 4381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:08.290084 4381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:08.291872 4381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "c320d9d3c1ae4d5f83b1e1f2a4579c2c"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "0dbbc9b05d00a5da5f86f1bbe7c6a8bc"
server_key_iv: "379f2c5adbd2c58f55e001c6bc71cf7a"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.292318 4381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "c320d9d3c1ae4d5f83b1e1f2a4579c2c"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "0dbbc9b05d00a5da5f86f1bbe7c6a8bc"
server_key_iv: "379f2c5adbd2c58f55e001c6bc71cf7a"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.295538 4381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:18:08.297644 4401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.298676 4381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:08.298763 4381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "c320d9d3c1ae4d5f83b1e1f2a4579c2c"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "0dbbc9b05d00a5da5f86f1bbe7c6a8bc"
server_key_iv: "379f2c5adbd2c58f55e001c6bc71cf7a"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.298859 4381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:08.315361 4381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:08.318281 4381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:08.318478 4381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:08.319043 4381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:08.319820 4381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:08.319891 4381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.319958 4381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:08.320004 4381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.328959 4381 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:35553
I20260808 18:18:08.328990 4574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:35553 every 8 connection(s)
I20260808 18:18:08.329706 4381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:08.337023 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 4381
I20260808 18:18:08.337147 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:08.337472 27723 external_mini_cluster.cc:1428] Setting key 2791e39a772a8ff075acdb91cdec8296
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:08.343123 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.331244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:55081 (local address 127.27.18.254:42265)
0808 18:18:08.331410 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.331415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.332251 (+ 836us) server_negotiation.cc:407] Connection header received
0808 18:18:08.333620 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.333624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.333683 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.333785 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.334852 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.335415 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.336498 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.336723 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.336869 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.339539 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.339546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.339551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.339581 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.341104 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.341699 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.341702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.341704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.341774 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.342145 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.342147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.342148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.342314 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.342383 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.342928 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.343016 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:18:08.344170 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.331686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42265 (local address 127.27.18.193:55081)
0808 18:18:08.332104 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:08.332144 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:08.333420 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:08.333952 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:08.333958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:08.334271 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:08.334673 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:08.334685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.335545 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.335555 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:08.336374 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:08.336383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.337230 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.337236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:08.337433 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.337945 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:08.337959 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:08.339342 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:08.341220 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:08.341227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:08.341241 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:08.341576 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:08.341867 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:08.341872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:08.341876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:08.342052 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:08.342410 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:08.342417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:08.342692 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 18:18:08.342892 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 18:18:08.343196 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
I20260808 18:18:08.345329 4576 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42265
I20260808 18:18:08.345567 4576 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:08.346135 4576 heartbeater.cc:511] Master 127.27.18.254:42265 requested a full tablet report, sending...
I20260808 18:18:08.347585 4288 ts_manager.cc:194] Registered new tserver with Master: c320d9d3c1ae4d5f83b1e1f2a4579c2c (127.27.18.193:35553)
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:18:08.348657 4288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:55081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:18:08.364634 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42265
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:08.469760 4584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:08.470024 4584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:08.470090 4584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:08.474030 4584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:08.474078 4584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:08.474131 4584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:08.479106 4584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:08.480052 4584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:08.481001 4584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:08.490332 4591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.490375 4594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.490355 4590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:08.490533 4584 server_base.cc:1034] running on GCE node
I20260808 18:18:08.490993 4584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:08.491705 4584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:08.492889 4584 hybrid_clock.cc:648] HybridClock initialized: now 1786213088492876 us; error 25 us; skew 500 ppm
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.495472 4584 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:18:08.496233 4584 webserver.cc:460] Webserver started at http://127.27.18.194:42767/ using document root <none> and password file <none>
I20260808 18:18:08.496752 4584 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:08.496851 4584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:08.497097 4584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:08.499011 4584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "cc58be2da81d40638eb8394b2a1d9752"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "c31445ae32310f708773956c3e79f3fa"
server_key_iv: "270904313f96fa6768523babfcdd13db"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.499620 4584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "cc58be2da81d40638eb8394b2a1d9752"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "c31445ae32310f708773956c3e79f3fa"
server_key_iv: "270904313f96fa6768523babfcdd13db"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.503321 4584 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:18:08.505641 4604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.506805 4584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:08.506896 4584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "cc58be2da81d40638eb8394b2a1d9752"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "c31445ae32310f708773956c3e79f3fa"
server_key_iv: "270904313f96fa6768523babfcdd13db"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.506968 4584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:08.542411 4584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:08.545010 4584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:08.545149 4584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:08.545655 4584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:08.546362 4584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:08.546406 4584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.546443 4584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:08.546464 4584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.554638 4584 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:38071
I20260808 18:18:08.554662 4774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:38071 every 8 connection(s)
I20260808 18:18:08.555616 4584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:08.560214 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 4584
I20260808 18:18:08.560338 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:08.560595 27723 external_mini_cluster.cc:1428] Setting key e93e6f84181b255aad59bf461453d9d0
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:08.569794 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.557327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:43975 (local address 127.27.18.254:42265)
0808 18:18:08.557458 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.557462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.558219 (+ 757us) server_negotiation.cc:407] Connection header received
0808 18:18:08.559582 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.559585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.559644 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.559747 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.560944 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.561631 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.562829 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.563096 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.563205 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.566000 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.566006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.566012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.566043 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.567656 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.568151 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.568157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.568162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.568267 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.568618 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.568625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.568626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.568903 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.569025 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.569454 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.569608 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:18:08.570343 4778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.557657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42265 (local address 127.27.18.194:43975)
0808 18:18:08.558088 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:08.558124 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:08.559330 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:08.559916 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:08.559922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:08.560255 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:08.560798 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:08.560818 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.561782 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.561785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:08.562695 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:08.562704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.563251 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.563256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:08.563482 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.564128 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:08.564153 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:08.565800 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:08.567770 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:08.567775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:08.567785 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:08.568026 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:08.568376 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:08.568379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:08.568380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:08.568519 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:08.569011 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:08.569016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:08.569211 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 18:18:08.569362 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 18:18:08.569590 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
I20260808 18:18:08.571435 4775 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42265
I20260808 18:18:08.571681 4775 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
I20260808 18:18:08.572232 4775 heartbeater.cc:511] Master 127.27.18.254:42265 requested a full tablet report, sending...
I20260808 18:18:08.573421 4288 ts_manager.cc:194] Registered new tserver with Master: cc58be2da81d40638eb8394b2a1d9752 (127.27.18.194:38071)
I20260808 18:18:08.573990 4288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:43975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:18:08.588186 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42265
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:40513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:08.691766 4787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:08.691968 4787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:08.692028 4787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:08.695655 4787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:08.695700 4787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:08.695803 4787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:08.700726 4787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:08.701649 4787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:08.702605 4787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:08.711602 4793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.711658 4798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:08.711759 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:08.711918 4787 server_base.cc:1034] running on GCE node
I20260808 18:18:08.712330 4787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:08.712924 4787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:08.714089 4787 hybrid_clock.cc:648] HybridClock initialized: now 1786213088714062 us; error 39 us; skew 500 ppm
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.716741 4787 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:18:08.717494 4787 webserver.cc:460] Webserver started at http://127.27.18.195:42007/ using document root <none> and password file <none>
I20260808 18:18:08.718007 4787 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:08.718065 4787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:08.718307 4787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:08.720193 4787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "8b478658102e4a069fd4d4113f08d487"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "7cbca209bd466c2ce17c6d20370b3699"
server_key_iv: "dead66a8eac9b3ea918b9cce6c556eab"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.720733 4787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "8b478658102e4a069fd4d4113f08d487"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "7cbca209bd466c2ce17c6d20370b3699"
server_key_iv: "dead66a8eac9b3ea918b9cce6c556eab"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.724216 4787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:08.726337 4809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.727455 4787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:08.727574 4787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "8b478658102e4a069fd4d4113f08d487"
format_stamp: "Formatted at 2026-08-08 18:18:08 on dist-test-slave-x4qh"
server_key: "7cbca209bd466c2ce17c6d20370b3699"
server_key_iv: "dead66a8eac9b3ea918b9cce6c556eab"
server_key_version: "encryptionkey@0"
I20260808 18:18:08.727661 4787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:08.738153 4787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:08.740597 4787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:08.740774 4787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:08.741284 4787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:08.742003 4787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:08.742046 4787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.742106 4787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:08.742141 4787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:08.750862 4787 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:44721
I20260808 18:18:08.750886 4972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:44721 every 8 connection(s)
I20260808 18:18:08.751657 4787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:08.753407 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 4787
I20260808 18:18:08.753517 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:08.753794 27723 external_mini_cluster.cc:1428] Setting key 56968823976c4606cb56470a1d211cb3
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:08.764904 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.753405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:44629 (local address 127.27.18.254:42265)
0808 18:18:08.753545 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.753551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.754451 (+ 900us) server_negotiation.cc:407] Connection header received
0808 18:18:08.755998 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.756002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.756058 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.756134 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.757177 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.757706 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.758789 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.759023 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.759132 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.761408 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.761412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.761415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.761440 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.762790 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.763328 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.763331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.763332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.763487 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.763910 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.763912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.763913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.764078 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.764155 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.764696 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.764787 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:18:08.765547 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.753789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42265 (local address 127.27.18.195:44629)
0808 18:18:08.754307 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:08.754359 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:08.755788 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:08.756284 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:08.756291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:08.756621 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:08.757039 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:08.757051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.757852 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.757862 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:08.758665 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:08.758674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.759169 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.759172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:08.759334 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.759912 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:08.759928 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:08.761246 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:08.762904 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:08.762910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:08.762922 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:08.763210 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:08.763580 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:08.763582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:08.763583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:08.763763 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:08.764184 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:08.764189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:08.764455 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 18:18:08.764643 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 18:18:08.764863 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":138,"threads_started":1}
I20260808 18:18:08.766678 4973 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42265
I20260808 18:18:08.766913 4973 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:08.767467 4973 heartbeater.cc:511] Master 127.27.18.254:42265 requested a full tablet report, sending...
I20260808 18:18:08.768422 4288 ts_manager.cc:194] Registered new tserver with Master: 8b478658102e4a069fd4d4113f08d487 (127.27.18.195:44721)
I20260808 18:18:08.769019 4288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:44629
I20260808 18:18:08.779124 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/test-admin.keytab.
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.794765 27723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 18:18:08.797422 4983 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:08.796063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.27.18.193:35553)
0808 18:18:08.796388 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.796396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.796433 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:18:08.796478 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.796482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.796638 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.796756 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.797210 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51204: BlockingRecv error: recv got EOF from 127.0.0.1:51204 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
W20260808 18:18:08.797514 27723 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.798476 27723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.799335 27723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:18:08.808173 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.800132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.27.18.193:35553)
0808 18:18:08.800283 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.800286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.800352 (+ 66us) server_negotiation.cc:407] Connection header received
0808 18:18:08.800434 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.800436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.800471 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.800533 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.800990 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.801755 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.802622 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.803010 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.803092 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.805240 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.805249 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.805260 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.805280 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.806622 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.806987 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.806992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.806993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.807064 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.807285 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.807287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.807288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.807572 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.807714 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.807884 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.808047 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 18:18:08.811851 4983 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:08.810897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.27.18.193:35553)
0808 18:18:08.811008 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.811012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.811113 (+ 101us) server_negotiation.cc:407] Connection header received
0808 18:18:08.811195 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.811197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.811272 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.811344 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.811752 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51228: BlockingRecv error: recv got EOF from 127.0.0.1:51228 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 18:18:08.811966 27723 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.812933 27723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.813875 27723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:18:08.822077 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.814619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.27.18.193:35553)
0808 18:18:08.814733 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.814736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.814841 (+ 105us) server_negotiation.cc:407] Connection header received
0808 18:18:08.814968 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.814971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.815012 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.815091 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.815621 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.816126 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.816872 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.817091 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.817196 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.819248 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.819258 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.819261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.819281 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.820498 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.820902 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.820906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.820908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.820986 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.821254 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.821257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.821259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.821442 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.821587 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.821836 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.821940 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 18:18:08.824884 4983 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:08.823767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.27.18.193:35553)
0808 18:18:08.823893 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.823911 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.824029 (+ 118us) server_negotiation.cc:407] Connection header received
0808 18:18:08.824136 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.824139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.824179 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.824258 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.824769 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51246: BlockingRecv error: recv got EOF from 127.0.0.1:51246 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:18:08.825040 27723 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786213058405439-27723-0/krb5kdc/krb5cc' of type FILE
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.825976 27723 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:08.826809 27723 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[4215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:18:08.834290 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:08.827547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.27.18.193:35553)
0808 18:18:08.827657 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:08.827661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:08.827792 (+ 131us) server_negotiation.cc:407] Connection header received
0808 18:18:08.827876 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:08.827878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:08.827908 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:08.827967 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:08.828385 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.828815 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.829491 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:08.829679 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:08.829746 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:08.831936 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:08.831939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:08.831940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:08.831959 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:08.833024 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.833367 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.833370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.833371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.833435 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:08.833649 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:08.833651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:08.833652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:08.833780 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:08.833887 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:08.834032 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 18:18:08.834136 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:18:08.835119 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 4381
I20260808 18:18:08.846577 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 4584
I20260808 18:18:08.860096 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 4787
I20260808 18:18:08.874456 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 4232
2026-08-08T18:18:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2292 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[5003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[5003](info): set up 2 sockets
Aug 08 18:18:08 dist-test-slave-x4qh krb5kdc[5003](info): commencing operation
krb5kdc: starting...
W20260808 18:18:10.052376 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab.
Entry for principal oryx/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab.
I20260808 18:18:10.164377 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42985
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab
--principal=oryx/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:10.266745 5020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:10.267022 5020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:10.267107 5020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:10.270840 5020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:10.270916 5020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:10.270953 5020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:10.271003 5020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:10.271034 5020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:10.276026 5020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab
--principal=oryx/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:10.276916 5020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:10.277788 5020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:10.285562 5028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:10.285581 5027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:10.285581 5034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.285627 5020 server_base.cc:1034] running on GCE node
I20260808 18:18:10.286346 5020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:10.287281 5020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:10.288445 5020 hybrid_clock.cc:648] HybridClock initialized: now 1786213090288419 us; error 37 us; skew 500 ppm
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:10.290936 5020 init.cc:377] Logged in from keytab as oryx/127.27.18.254@KRBTEST.COM (short username oryx)
I20260808 18:18:10.293421 5020 webserver.cc:460] Webserver started at http://127.27.18.254:41403/ using document root <none> and password file <none>
I20260808 18:18:10.293933 5020 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:10.294023 5020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:10.294243 5020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:10.296254 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "5af716cd9591425f8195a786bc9183a8"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "69d0b13106078df39a97f59add3736fb"
server_key_iv: "626c35c026efd93232dd7e6138527be5"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.296798 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "5af716cd9591425f8195a786bc9183a8"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "69d0b13106078df39a97f59add3736fb"
server_key_iv: "626c35c026efd93232dd7e6138527be5"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.300235 5020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:10.302564 5044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.303735 5020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:18:10.303869 5020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5af716cd9591425f8195a786bc9183a8"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "69d0b13106078df39a97f59add3736fb"
server_key_iv: "626c35c026efd93232dd7e6138527be5"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.303972 5020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:10.323597 5020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:10.326190 5020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:10.326382 5020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:10.333047 5020 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:42985
I20260808 18:18:10.333348 5120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:42985 every 8 connection(s)
I20260808 18:18:10.333966 5020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:10.336675 5122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:10.340077 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 5020
I20260808 18:18:10.340227 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:10.340499 27723 external_mini_cluster.cc:1428] Setting key 43fa9b1b2c2da7d9b0bddfb0f71d1cd1
I20260808 18:18:10.342214 5122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Bootstrap starting.
I20260808 18:18:10.344370 5122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:10.345080 5122 log.cc:824] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:10.347095 5122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: No bootstrap required, opened a new log
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:10.351219 5122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } }
I20260808 18:18:10.351497 5122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:10.351578 5122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af716cd9591425f8195a786bc9183a8, State: Initialized, Role: FOLLOWER
I20260808 18:18:10.351960 5122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } }
I20260808 18:18:10.352106 5122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:10.352198 5122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:10.352312 5122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:10.353305 5122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } }
I20260808 18:18:10.353663 5122 leader_election.cc:302] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af716cd9591425f8195a786bc9183a8; no voters:
I20260808 18:18:10.354032 5122 leader_election.cc:288] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:10.354126 5135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:10.354338 5135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 LEADER]: Becoming Leader. State: Replica: 5af716cd9591425f8195a786bc9183a8, State: Running, Role: LEADER
I20260808 18:18:10.354734 5135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } }
I20260808 18:18:10.355098 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:10.341766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55994 (local address 127.27.18.254:42985)
0808 18:18:10.342181 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:10.342195 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:10.342227 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:18:10.342885 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:10.342900 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:10.343187 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:10.343537 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:10.344315 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.345427 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.346654 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.347099 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.347281 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:10.350059 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:10.350064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:10.350075 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:10.350102 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:10.352062 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:10.352595 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:10.352600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:10.352605 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:10.352709 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:10.352978 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:10.352980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:10.352982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:10.353301 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:10.353472 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:10.353714 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 18:18:10.354045 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":133,"threads_started":1}
I20260808 18:18:10.355214 5122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:10.356460 5139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } }
I20260808 18:18:10.356576 5139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:10.356801 5139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af716cd9591425f8195a786bc9183a8. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } }
I20260808 18:18:10.356870 5139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:10.357897 5149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:10.360630 5149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:10.366582 5149 catalog_manager.cc:1347] Generated new cluster ID: e585fb888d9840859859ff3058b74515
I20260808 18:18:10.366641 5149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:10.376377 5149 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:10.377411 5149 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:10.385329 5149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Generated new TSK 0
I20260808 18:18:10.385959 5149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab.
Entry for principal oryx/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab.
I20260808 18:18:10.413559 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42985
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:10.519559 5169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:10.519748 5169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:10.519783 5169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:10.523183 5169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:10.523222 5169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:10.523269 5169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:10.527848 5169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:10.528715 5169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:10.529498 5169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:10.538272 5176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.538501 5169 server_base.cc:1034] running on GCE node
W20260808 18:18:10.538291 5179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:10.538305 5177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.538988 5169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:10.539588 5169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:10.540760 5169 hybrid_clock.cc:648] HybridClock initialized: now 1786213090540739 us; error 35 us; skew 500 ppm
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:10.543298 5169 init.cc:377] Logged in from keytab as oryx/127.27.18.193@KRBTEST.COM (short username oryx)
I20260808 18:18:10.544070 5169 webserver.cc:460] Webserver started at http://127.27.18.193:42625/ using document root <none> and password file <none>
I20260808 18:18:10.544544 5169 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:10.544600 5169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:10.544744 5169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:10.546499 5169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "46db5413c4c9d8eec104f16b0b40f29a"
server_key_iv: "e858ff6c5558f88989fa18b1630e92d6"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.546990 5169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "46db5413c4c9d8eec104f16b0b40f29a"
server_key_iv: "e858ff6c5558f88989fa18b1630e92d6"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.550233 5169 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:10.552361 5188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.553367 5169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:10.553450 5169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "46db5413c4c9d8eec104f16b0b40f29a"
server_key_iv: "e858ff6c5558f88989fa18b1630e92d6"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.553522 5169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:10.573503 5169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:10.576368 5169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:10.576502 5169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:10.576954 5169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:10.577687 5169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:10.577744 5169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.577790 5169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:10.577826 5169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.586061 5169 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:37223
I20260808 18:18:10.586128 5353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:37223 every 8 connection(s)
I20260808 18:18:10.586864 5169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:10.589121 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 5169
I20260808 18:18:10.589267 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:10.589603 27723 external_mini_cluster.cc:1428] Setting key 6cf17e39eee3f2c4eb2edb41216ad8b0
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:10.601069 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:10.588539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:42675 (local address 127.27.18.254:42985)
0808 18:18:10.588702 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:10.588706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:10.589498 (+ 792us) server_negotiation.cc:407] Connection header received
0808 18:18:10.591120 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:10.591122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:10.591196 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:10.591275 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:10.592808 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.593314 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.594375 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.594604 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.594705 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:10.597499 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:10.597502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:10.597504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:10.597525 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:10.598877 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:10.599551 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:10.599554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:10.599555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:10.599623 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:10.600028 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:10.600030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:10.600031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:10.600191 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:10.600260 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:10.600866 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 18:18:10.600960 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for oryx/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:18:10.601964 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:10.588832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:42675)
0808 18:18:10.589329 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:10.589386 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:10.590896 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:10.591502 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:10.591528 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:10.591995 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:10.592631 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:10.592666 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.593446 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.593450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:10.594246 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:10.594255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.594752 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.594756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:10.594998 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:10.595647 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:10.595672 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:10.597354 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:10.598994 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:10.599003 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:10.599018 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:10.599448 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:10.599724 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:10.599729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:10.599732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:10.599929 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:10.600295 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:10.600301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:10.600591 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 18:18:10.600822 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 18:18:10.601161 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
I20260808 18:18:10.602993 5354 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42985
I20260808 18:18:10.603295 5354 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:10.603752 5354 heartbeater.cc:511] Master 127.27.18.254:42985 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.194@KRBTEST.COM" created.
I20260808 18:18:10.605326 5069 ts_manager.cc:194] Registered new tserver with Master: c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223)
I20260808 18:18:10.606360 5069 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:42675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab.
Entry for principal oryx/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab.
I20260808 18:18:10.618053 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42985
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:10.725272 5365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:10.725483 5365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:10.725521 5365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:10.729296 5365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:10.729338 5365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:10.729386 5365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:10.734175 5365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:10.735026 5365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:10.735944 5365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:10.744664 5376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:10.744740 5375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.744931 5365 server_base.cc:1034] running on GCE node
W20260808 18:18:10.745164 5378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.745534 5365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:10.746133 5365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:10.747289 5365 hybrid_clock.cc:648] HybridClock initialized: now 1786213090747285 us; error 30 us; skew 500 ppm
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:10.749842 5365 init.cc:377] Logged in from keytab as oryx/127.27.18.194@KRBTEST.COM (short username oryx)
I20260808 18:18:10.750636 5365 webserver.cc:460] Webserver started at http://127.27.18.194:37965/ using document root <none> and password file <none>
I20260808 18:18:10.751107 5365 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:10.751189 5365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:10.751349 5365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:10.753222 5365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "d26019add54840f7a3bd87db4626333c"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "cd9c8d6d20e3eb73456afe53e48145a5"
server_key_iv: "0553d30120543e47558bab4f90cb09ac"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.753690 5365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "d26019add54840f7a3bd87db4626333c"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "cd9c8d6d20e3eb73456afe53e48145a5"
server_key_iv: "0553d30120543e47558bab4f90cb09ac"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.757547 5365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:18:10.759858 5388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.761011 5365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:10.761143 5365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "d26019add54840f7a3bd87db4626333c"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "cd9c8d6d20e3eb73456afe53e48145a5"
server_key_iv: "0553d30120543e47558bab4f90cb09ac"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.761242 5365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:10.793721 5365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:10.796783 5365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:10.796957 5365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:10.797482 5365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:10.798220 5365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:10.798264 5365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.798331 5365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:10.798362 5365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.806893 5365 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33819
I20260808 18:18:10.806917 5554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33819 every 8 connection(s)
I20260808 18:18:10.807801 5365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:10.813961 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 5365
I20260808 18:18:10.814103 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:10.814407 27723 external_mini_cluster.cc:1428] Setting key e7b6a7470ac9c1596f40d479ceab6f8f
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:10.821353 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:10.809485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:48201 (local address 127.27.18.254:42985)
0808 18:18:10.809613 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:10.809617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:10.810562 (+ 945us) server_negotiation.cc:407] Connection header received
0808 18:18:10.811976 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:10.811979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:10.812049 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:10.812147 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:10.813148 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.813666 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.815097 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.815447 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.815549 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:10.817821 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:10.817826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:10.817830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:10.817864 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:10.819147 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:10.819694 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:10.819700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:10.819705 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:10.819796 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:10.820096 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:10.820102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:10.820106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:10.820376 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:10.820555 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:10.820978 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 18:18:10.821118 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:18:10.821918 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:10.809758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:48201)
0808 18:18:10.810370 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:10.810434 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:10.811754 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:10.812303 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:10.812309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:10.812604 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:10.813009 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:10.813020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.813824 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.813826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:10.814991 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:10.815005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:10.815610 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:10.815614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:10.815759 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:10.816380 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:10.816394 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:10.817665 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:10.819263 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:10.819271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:10.819286 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:10.819599 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:10.819890 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:10.819892 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:10.819893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:10.820008 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:10.820522 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:10.820526 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:10.820736 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 18:18:10.820877 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 18:18:10.821084 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
I20260808 18:18:10.822925 5555 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42985
I20260808 18:18:10.823146 5555 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:10.823608 5555 heartbeater.cc:511] Master 127.27.18.254:42985 requested a full tablet report, sending...
I20260808 18:18:10.824579 5069 ts_manager.cc:194] Registered new tserver with Master: d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
WARNING: no policy specified for oryx/127.27.18.195@KRBTEST.COM; defaulting to no policy
I20260808 18:18:10.825273 5069 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:48201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab.
Entry for principal oryx/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab.
I20260808 18:18:10.840420 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42985
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:10.944083 5560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:10.944319 5560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:10.944416 5560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:10.948093 5560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:10.948161 5560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:10.948237 5560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:10.952792 5560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:42985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:10.953614 5560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:10.954457 5560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:10.963522 5567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:10.963512 5566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.963512 5560 server_base.cc:1034] running on GCE node
W20260808 18:18:10.963493 5570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:10.964150 5560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:10.964710 5560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:10.965893 5560 hybrid_clock.cc:648] HybridClock initialized: now 1786213090965883 us; error 46 us; skew 500 ppm
Aug 08 18:18:10 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:10.968509 5560 init.cc:377] Logged in from keytab as oryx/127.27.18.195@KRBTEST.COM (short username oryx)
I20260808 18:18:10.969218 5560 webserver.cc:460] Webserver started at http://127.27.18.195:37875/ using document root <none> and password file <none>
I20260808 18:18:10.969689 5560 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:10.969743 5560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:10.969880 5560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:10.971769 5560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "02e9a23d4880419185b4b55055c86ac4"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "11f533c1f5053dba8f487a3b24c8636b"
server_key_iv: "408cca2dd31431ec1b9aa1682f86857e"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.972259 5560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "02e9a23d4880419185b4b55055c86ac4"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "11f533c1f5053dba8f487a3b24c8636b"
server_key_iv: "408cca2dd31431ec1b9aa1682f86857e"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.975629 5560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:18:10.977622 5578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:10.978657 5560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:10.978762 5560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "02e9a23d4880419185b4b55055c86ac4"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "11f533c1f5053dba8f487a3b24c8636b"
server_key_iv: "408cca2dd31431ec1b9aa1682f86857e"
server_key_version: "encryptionkey@0"
I20260808 18:18:10.978870 5560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.004622 5560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:11.007705 5560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:11.007891 5560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:11.008411 5560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:11.009111 5560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:11.009155 5560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.009209 5560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:11.009239 5560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.017208 5560 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42621
I20260808 18:18:11.017236 5750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42621 every 8 connection(s)
I20260808 18:18:11.017987 5560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:11.026032 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 5560
I20260808 18:18:11.026135 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:11.026439 27723 external_mini_cluster.cc:1428] Setting key 3bdf19ebdf2f1790a56250110ee24941
I20260808 18:18:11.030095 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.019585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:52245 (local address 127.27.18.254:42985)
0808 18:18:11.019733 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.019737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.020533 (+ 796us) server_negotiation.cc:407] Connection header received
0808 18:18:11.021766 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.021768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.021813 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.021896 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.022800 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.023272 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.024320 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.024543 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.024617 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.026946 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.026950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.026952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.026996 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.028279 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.028734 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.028737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.028739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.028810 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.029132 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.029136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.029138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.029293 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.029406 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.029842 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.029946 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:18:11.030710 5754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.019980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:52245)
0808 18:18:11.020386 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.020423 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.021577 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.022047 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.022053 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.022322 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.022686 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.022696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.023428 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.023431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.024210 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.024219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.024661 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.024664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.024783 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.025308 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.025323 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.026814 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.028397 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.028402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.028414 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.028638 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.028896 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.028899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.028900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.029041 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.029407 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.029414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.029626 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.029783 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.030068 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
I20260808 18:18:11.031626 5752 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42985
I20260808 18:18:11.031834 5752 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:11.032312 5752 heartbeater.cc:511] Master 127.27.18.254:42985 requested a full tablet report, sending...
I20260808 18:18:11.033234 5069 ts_manager.cc:194] Registered new tserver with Master: 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195:42621)
I20260808 18:18:11.033776 5069 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:52245
I20260808 18:18:11.039804 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.18.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.253.keytab.
Entry for principal oryx/127.27.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.253.keytab.
I20260808 18:18:11.066468 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:39829
--webserver_interface=127.27.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.253.keytab
--principal=oryx/127.27.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42985,127.27.18.253:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:11.168462 5760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:11.168659 5760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:11.168694 5760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:11.172151 5760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:11.172191 5760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:11.172209 5760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:11.172223 5760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:11.172237 5760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:11.176798 5760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42985,127.27.18.253:39829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.253.keytab
--principal=oryx/127.27.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
W20260808 18:18:11.176954 5760 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.27.18.254:42985,127.27.18.253:39829'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 18:18:11.177585 5760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:11.178455 5760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:11.186148 5773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.186159 5769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.186164 5770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.186378 5760 server_base.cc:1034] running on GCE node
I20260808 18:18:11.186882 5760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:11.187885 5760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:11.189075 5760 hybrid_clock.cc:648] HybridClock initialized: now 1786213091189062 us; error 23 us; skew 500 ppm
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:11.191646 5760 init.cc:377] Logged in from keytab as oryx/127.27.18.253@KRBTEST.COM (short username oryx)
I20260808 18:18:11.192327 5760 webserver.cc:460] Webserver started at http://127.27.18.253:45895/ using document root <none> and password file <none>
I20260808 18:18:11.192746 5760 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.192796 5760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.192934 5760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:11.194555 5760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/instance:
uuid: "bc0fd7748bd9456bbd69c146e657ee08"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "3d84d818315339a468c10bb0574d46c9"
server_key_iv: "2e012f92c3aabb82631dbd625a82c5cc"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.195029 5760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal/instance:
uuid: "bc0fd7748bd9456bbd69c146e657ee08"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "3d84d818315339a468c10bb0574d46c9"
server_key_iv: "2e012f92c3aabb82631dbd625a82c5cc"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.198339 5760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:11.200464 5780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.201512 5760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:11.201632 5760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
uuid: "bc0fd7748bd9456bbd69c146e657ee08"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "3d84d818315339a468c10bb0574d46c9"
server_key_iv: "2e012f92c3aabb82631dbd625a82c5cc"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.201735 5760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.213176 5760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:11.215708 5760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:11.215899 5760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:11.222133 5760 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.253:39829
I20260808 18:18:11.222409 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.253:39829 every 8 connection(s)
I20260808 18:18:11.223042 5760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
I20260808 18:18:11.225598 5869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:11.229007 5869 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:18:11.231702 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 5760
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.253@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
I20260808 18:18:11.241897 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.230619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.27.18.254:42985)
0808 18:18:11.230794 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.230799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.231602 (+ 803us) server_negotiation.cc:407] Connection header received
0808 18:18:11.232608 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.232613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.232678 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.232765 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.233804 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.234342 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.235627 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.235844 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.235952 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.238419 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.238423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.238425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.238452 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.239940 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.240476 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.240479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.240481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.240558 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.240881 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.240885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.240886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.241092 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.241225 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.241625 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.241736 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 18:18:11.242595 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.231006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56000)
0808 18:18:11.231472 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.231511 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.232388 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.232980 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.232986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.233256 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.233659 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.233671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.234476 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.234479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.235497 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.235507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.235987 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.235992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.236180 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.236732 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.236756 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.238290 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.240045 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.240056 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.240064 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.240356 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.240645 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.240649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.240650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.240789 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.241224 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.241230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.241423 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.241578 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.241851 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":186,"spinlock_wait_cycles":1792,"thread_start_us":109,"threads_started":1}
I20260808 18:18:11.242595 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.232505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.27.18.253:39829)
0808 18:18:11.232842 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.232853 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.232911 (+ 58us) server_negotiation.cc:407] Connection header received
0808 18:18:11.233066 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.233072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.233224 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.233395 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.234149 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.235033 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.235915 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.236307 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.236436 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.238818 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.238823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.238831 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.238850 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.240318 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.240749 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.240753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.240754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.240824 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.241090 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.241093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.241094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.241412 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.241602 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.241805 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.241933 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":256,"thread_start_us":168,"threads_started":1}
I20260808 18:18:11.243818 5869 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.253@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
I20260808 18:18:11.251058 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.244236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:38942)
0808 18:18:11.244360 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.244374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.244444 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.244668 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.244673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.244833 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.244914 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.244919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.245561 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.245563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.246026 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.246032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.246941 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.246944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.247009 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.247345 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.247356 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.248435 (+ 1079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.250010 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.250012 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.250013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.250168 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.250386 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.250388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.250389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.250452 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.250723 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.250725 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.250805 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.250878 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.250957 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 18:18:11.251124 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.244328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.27.18.253:39829)
0808 18:18:11.244440 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.244442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.244453 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:18:11.244514 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.244519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.244564 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.244636 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.245014 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.245465 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.246131 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.246844 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.246923 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.248550 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.248552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.248554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.248574 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.249924 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.250250 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.250252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.250253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.250314 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.250522 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.250524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.250525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.250652 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.250716 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.250934 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.251012 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:18:11.253580 5869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Bootstrap starting.
I20260808 18:18:11.255589 5869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:11.256269 5869 log.cc:824] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:11.257943 5869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: No bootstrap required, opened a new log
I20260808 18:18:11.260818 5869 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } }
I20260808 18:18:11.260995 5869 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:11.261066 5869 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0fd7748bd9456bbd69c146e657ee08, State: Initialized, Role: FOLLOWER
I20260808 18:18:11.261441 5869 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } }
I20260808 18:18:11.261837 5878 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } } }
I20260808 18:18:11.261938 5878 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:11.262451 5869 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:18:11.266894 5892 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:18:11.267002 5892 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:18:11.269502 5760 master_runner.cc:428] Detected that this master bc0fd7748bd9456bbd69c146e657ee08 is joining an existing cluster
I20260808 18:18:11.269562 5760 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.18.253:39829
I20260808 18:18:11.279423 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.271639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56016)
0808 18:18:11.271918 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.271932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.272026 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.272376 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.272379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.272654 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.272797 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.272804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.273502 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.273507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.274111 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.274119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.274557 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.274561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.274661 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.275169 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.275192 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.275621 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.277834 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.277836 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.277837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.278037 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.278317 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.278319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.278321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.278402 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.278829 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.278832 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.278918 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.279079 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.279202 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260808 18:18:11.279490 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.271689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.27.18.254:42985)
0808 18:18:11.271818 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.271822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.272067 (+ 245us) server_negotiation.cc:407] Connection header received
0808 18:18:11.272110 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.272113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.272168 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.272260 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.272892 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.273389 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.274230 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.274437 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.274545 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.275766 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.275770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.275772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.275797 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.277510 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.278143 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.278146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.278147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.278223 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.278506 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.278510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.278511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.278717 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.278821 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.279144 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.279269 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:11.280496 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.272068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:38958)
0808 18:18:11.272351 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.272364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.272520 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.272821 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.272824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.273025 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.273156 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.273164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.274146 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.274162 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.275045 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.275056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.275685 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.275689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.275844 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.276374 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.276388 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.276779 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.278683 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.278685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.278687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.278935 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.279247 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.279251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.279253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.279333 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.279988 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.279991 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.280098 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.280196 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.280312 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":29,"thread_start_us":89,"threads_started":1}
I20260808 18:18:11.280501 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.272304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.27.18.253:39829)
0808 18:18:11.272395 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.272399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.272490 (+ 91us) server_negotiation.cc:407] Connection header received
0808 18:18:11.272649 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.272651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.272694 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.272786 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.273274 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.274027 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.275202 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.275557 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.275680 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.276918 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.276921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.276923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.276943 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.278570 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.279056 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.279059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.279061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.279139 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.279458 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.279461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.279463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.279664 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.279741 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.280267 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.280374 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":38}
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.254@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
I20260808 18:18:11.291265 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.283570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:38974)
0808 18:18:11.283849 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.283890 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.284019 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.284274 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.284277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.284574 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.284662 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.284667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.285553 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.285555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.286132 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.286152 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.286641 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.286643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.286711 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.287124 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.287134 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.288418 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.289991 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.289999 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.290002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.290197 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.290456 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.290458 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.290460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.290526 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.290868 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.290872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.290954 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.291052 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.291166 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260808 18:18:11.291344 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.283581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.27.18.253:39829)
0808 18:18:11.283682 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.283686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.283975 (+ 289us) server_negotiation.cc:407] Connection header received
0808 18:18:11.284136 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.284139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.284182 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.284257 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.284770 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.285450 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.286248 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.286544 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.286631 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.288544 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.288548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.288550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.288574 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.289886 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.290284 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.290287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.290288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.290371 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.290606 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.290609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.290611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.290776 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.290861 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.291122 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.291216 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:18:11.292443 5069 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.18.253:39829: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08"
member_type: NON_VOTER
last_known_addr {
host: "127.27.18.253"
port: 39829
}
attrs {
promote: true
}
}
dest_uuid: "5af716cd9591425f8195a786bc9183a8"
cas_config_opid_index: -1
I20260808 18:18:11.293016 5069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } }
I20260808 18:18:11.294394 5903 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bc0fd7748bd9456bbd69c146e657ee08 (127.27.18.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } }
I20260808 18:18:11.295034 5823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:11.296078 5903 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.18.253:39829
I20260808 18:18:11.296356 5823 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Refusing update from remote peer 5af716cd9591425f8195a786bc9183a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 18:18:11.296475 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } }
I20260808 18:18:11.296562 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:11.296723 5760 master.cc:537] Master@127.27.18.253:39829 shutting down...
I20260808 18:18:11.297194 5902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:18:11.299224 5045 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 18:18:11.300176 5045 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 -> Peer bc0fd7748bd9456bbd69c146e657ee08 (127.27.18.253:39829): Couldn't send request to peer bc0fd7748bd9456bbd69c146e657ee08. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 18:18:11.303041 5760 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:11.303411 5760 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:11.303591 5760 tablet_replica.cc:330] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: stopping tablet replica
WARNING: no policy specified for oryx/127.27.18.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.252@KRBTEST.COM" created.
I20260808 18:18:11.318909 5760 master.cc:552] Master@127.27.18.253:39829 shutdown complete.
I20260808 18:18:11.318959 5760 master_runner.cc:305] Clearing existing system tablet
I20260808 18:18:11.319193 5760 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.319247 5760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.320747 5760 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.321648 5910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.321913 5760 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.322052 5760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.322093 5760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
uuid: "bc0fd7748bd9456bbd69c146e657ee08"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "3d84d818315339a468c10bb0574d46c9"
server_key_iv: "2e012f92c3aabb82631dbd625a82c5cc"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.323992 5760 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.252.keytab.
Entry for principal oryx/127.27.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.252.keytab.
I20260808 18:18:11.325958 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:38281
--webserver_interface=127.27.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.252.keytab
--principal=oryx/127.27.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:18:11.326565 5760 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 18:18:11.326646 5760 log.cc:1196] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 18:18:11.326910 5760 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Deleting consensus metadata
I20260808 18:18:11.327062 5760 master_runner.cc:315] Copying system tablet from 127.27.18.254:42985
I20260808 18:18:11.328325 5760 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: tablet copy: Beginning tablet copy session from remote peer at address 127.27.18.254:42985
I20260808 18:18:11.343616 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.331726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.27.18.254:42985)
0808 18:18:11.331856 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.331862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.335651 (+ 3789us) server_negotiation.cc:407] Connection header received
0808 18:18:11.335848 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.335852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.335899 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.335948 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.336571 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.337370 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.338574 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.338867 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.338932 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.340341 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.340345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.340348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.340371 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.341897 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.342411 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.342414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.342416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.342483 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.342751 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.342754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.342756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.342928 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.342999 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.343337 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.343470 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:18:11.344239 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.331679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56032)
0808 18:18:11.335557 (+ 3878us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.335571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.335725 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.336082 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.336084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.336323 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.336448 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.336455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.337508 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.337511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.338453 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.338464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.339076 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.339079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.339168 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.339718 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.339732 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.340187 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.342020 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.342022 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.342024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.342284 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.342595 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.342598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.342600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.342669 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.343077 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.343080 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.343173 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.343931 (+ 758us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.344064 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3803,"thread_start_us":80,"threads_started":1}
I20260808 18:18:11.345003 5097 tablet_copy_service.cc:140] P 5af716cd9591425f8195a786bc9183a8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bc0fd7748bd9456bbd69c146e657ee08 ({username='oryx', principal='oryx/127.27.18.253@KRBTEST.COM'} at 127.0.0.1:56032)
I20260808 18:18:11.345157 5097 tablet_copy_service.cc:161] P 5af716cd9591425f8195a786bc9183a8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bc0fd7748bd9456bbd69c146e657ee08 at {username='oryx', principal='oryx/127.27.18.253@KRBTEST.COM'} at 127.0.0.1:56032: session id = bc0fd7748bd9456bbd69c146e657ee08-00000000000000000000000000000000
I20260808 18:18:11.346922 5097 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 18:18:11.348624 5760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 18:18:11.350795 5760 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: tablet copy: Starting download of 0 data blocks...
I20260808 18:18:11.351006 5760 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: tablet copy: Starting download of 1 WAL segments...
I20260808 18:18:11.352677 5760 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 18:18:11.354022 5097 tablet_copy_service.cc:342] P 5af716cd9591425f8195a786bc9183a8: Request end of tablet copy session bc0fd7748bd9456bbd69c146e657ee08-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.18.253@KRBTEST.COM'} at 127.0.0.1:56032
I20260808 18:18:11.354126 5097 tablet_copy_service.cc:433] P 5af716cd9591425f8195a786bc9183a8: ending tablet copy session bc0fd7748bd9456bbd69c146e657ee08-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bc0fd7748bd9456bbd69c146e657ee08
I20260808 18:18:11.361554 5760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:11.363071 5921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.363162 5926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.363611 5760 server_base.cc:1034] running on GCE node
W20260808 18:18:11.367527 5923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.367744 5760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:11.368067 5760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:11.369203 5760 hybrid_clock.cc:648] HybridClock initialized: now 1786213091369190 us; error 30 us; skew 500 ppm
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:11.370632 5760 init.cc:377] Logged in from keytab as oryx/127.27.18.253@KRBTEST.COM (short username oryx)
I20260808 18:18:11.371089 5760 webserver.cc:460] Webserver started at http://127.27.18.253:33855/ using document root <none> and password file <none>
I20260808 18:18:11.371269 5760 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.371335 5760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.372465 5760 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.373364 5940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.373588 5760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.373684 5760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
uuid: "bc0fd7748bd9456bbd69c146e657ee08"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "3d84d818315339a468c10bb0574d46c9"
server_key_iv: "2e012f92c3aabb82631dbd625a82c5cc"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.373766 5760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.389170 5760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:11.391050 5760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:11.396205 5760 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.253:39829
I20260808 18:18:11.396756 5760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
I20260808 18:18:11.396226 6024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.253:39829 every 8 connection(s)
I20260808 18:18:11.398407 6025 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:18:11.399196 6025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Bootstrap starting.
I20260808 18:18:11.407106 6025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:18:11.407828 6025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Bootstrap complete.
I20260808 18:18:11.408320 6025 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } }
I20260808 18:18:11.408442 6025 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bc0fd7748bd9456bbd69c146e657ee08, State: Initialized, Role: LEARNER
I20260808 18:18:11.408556 6025 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } }
I20260808 18:18:11.408854 6030 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } }
I20260808 18:18:11.409000 6030 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:18:11.409008 6025 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:11.412111 6046 catalog_manager.cc:1259] Loaded cluster ID: e585fb888d9840859859ff3058b74515
I20260808 18:18:11.412178 6046 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: loading cluster ID for follower catalog manager: success
I20260808 18:18:11.413568 6046 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: acquiring CA information for follower catalog manager: success
I20260808 18:18:11.414533 6046 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 18:18:11.473282 5911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:11.473496 5911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:11.473532 5911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:11.477429 5911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:11.477476 5911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:11.477494 5911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:11.477509 5911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:11.477522 5911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:11.482028 5911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.252.keytab
--principal=oryx/127.27.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:38281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:11.482908 5911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:11.483908 5911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:11.491611 6056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.491719 6051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.491874 5911 server_base.cc:1034] running on GCE node
W20260808 18:18:11.491717 6053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.492496 5911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:11.493505 5911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:11.494690 5911 hybrid_clock.cc:648] HybridClock initialized: now 1786213091494659 us; error 43 us; skew 500 ppm
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:11.497282 5911 init.cc:377] Logged in from keytab as oryx/127.27.18.252@KRBTEST.COM (short username oryx)
I20260808 18:18:11.498044 5911 webserver.cc:460] Webserver started at http://127.27.18.252:36151/ using document root <none> and password file <none>
I20260808 18:18:11.498538 5911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.498620 5911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.498781 5911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:11.500605 5911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/instance:
uuid: "0ae25ea8906d4eef916298529eaec76d"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "877e08501323784f94bca78948b11539"
server_key_iv: "d09f5760bde0ff77af59a146cf8de3a1"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.501084 5911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal/instance:
uuid: "0ae25ea8906d4eef916298529eaec76d"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "877e08501323784f94bca78948b11539"
server_key_iv: "d09f5760bde0ff77af59a146cf8de3a1"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.504312 5911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 18:18:11.507167 6068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.508502 5911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:11.508618 5911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
uuid: "0ae25ea8906d4eef916298529eaec76d"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "877e08501323784f94bca78948b11539"
server_key_iv: "d09f5760bde0ff77af59a146cf8de3a1"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.508741 5911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.528563 5911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:11.531065 5911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:11.531267 5911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:11.538090 5911 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.252:38281
I20260808 18:18:11.538386 6140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.252:38281 every 8 connection(s)
I20260808 18:18:11.539008 5911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
I20260808 18:18:11.541707 6141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:11.543013 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 5911
I20260808 18:18:11.545897 6141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.18.252@KRBTEST.COM
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.252@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:11.556774 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.547573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.27.18.254:42985)
0808 18:18:11.547720 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.547725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.547977 (+ 252us) server_negotiation.cc:407] Connection header received
0808 18:18:11.548144 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.548157 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.548200 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.548285 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.548912 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.549424 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.550449 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.550839 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.550960 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.553286 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.553290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.553292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.553319 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.554749 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.555349 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.555352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.555353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.555479 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.555743 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.555746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.555748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.555959 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.556051 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.556496 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.556609 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:18:11.557529 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.547431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56038)
0808 18:18:11.547859 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.547894 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.548035 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.548337 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.548342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.548649 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.548768 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.548773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.549557 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.549560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.550305 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.550318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.550993 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.550995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.551074 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.551595 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.551610 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.553155 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.554853 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.554863 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.554873 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.555245 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.555574 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.555576 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.555578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.555650 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.556059 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.556065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.556269 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.556445 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.556789 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":36,"thread_start_us":175,"threads_started":1}
I20260808 18:18:11.557533 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.544462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.27.18.252:38281)
0808 18:18:11.544888 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.544902 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.544933 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:18:11.545567 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.545583 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.545875 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.546184 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.546984 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.548357 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.549180 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.549622 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.549798 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.552659 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.552665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.552686 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.552707 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.554510 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.554944 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.554948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.554949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.555071 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.555436 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.555442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.555443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.555904 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.556127 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.556473 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.556715 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":290,"thread_start_us":131,"threads_started":1}
I20260808 18:18:11.558692 6141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.252@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
I20260808 18:18:11.566382 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.559123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:38976)
0808 18:18:11.559255 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.559264 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.559322 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.559730 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.559733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.559870 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.559945 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.559949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.560938 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.560941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.561590 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.561599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.562018 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.562020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.562089 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.562533 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.562548 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.563653 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.565199 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.565202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.565203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.565444 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.565680 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.565682 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.565683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.565753 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.566065 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.566067 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.566133 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.566215 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.566293 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 18:18:11.566620 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.559235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.27.18.253:39829)
0808 18:18:11.559518 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.559524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.559538 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:11.559578 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.559580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.559625 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.559718 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.560058 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.560791 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.561693 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.561922 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.562001 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.563766 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.563769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.563771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.563790 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.565105 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.565527 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.565530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.565531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.565602 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.565837 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.565839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.565840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.565987 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.566058 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.566268 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.566357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260808 18:18:11.567219 6141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.252@KRBTEST.COM for oryx/127.27.18.252@KRBTEST.COM
I20260808 18:18:11.574307 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.567673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35660)
0808 18:18:11.567746 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.567759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.567833 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.567996 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.567998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.568103 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.568162 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.568165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.569175 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.569177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.569653 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.569660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.570115 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.570117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.570167 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.570596 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.570611 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.571679 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.573120 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.573122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.573123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.573276 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.573505 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.573507 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.573508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.573570 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.573919 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.573922 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.574006 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.574078 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.574173 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 18:18:11.574410 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.567694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.27.18.252:38281)
0808 18:18:11.567792 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.567795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.567820 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:18:11.567884 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.567885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.567914 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.567969 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.568247 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.568685 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.569770 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.570027 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.570097 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.571790 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.571793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.571794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.571820 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.573037 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.573358 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.573360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.573361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.573424 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.573640 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.573642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.573642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.573791 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.573867 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.574138 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.574216 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:18:11.576965 6141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Bootstrap starting.
I20260808 18:18:11.578989 6141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:11.579819 6141 log.cc:824] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:11.581481 6141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: No bootstrap required, opened a new log
I20260808 18:18:11.584508 6141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } }
I20260808 18:18:11.584719 6141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:11.584800 6141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae25ea8906d4eef916298529eaec76d, State: Initialized, Role: FOLLOWER
I20260808 18:18:11.585240 6141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } }
I20260808 18:18:11.585656 6147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } } }
I20260808 18:18:11.585759 6147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:11.586437 6141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:18:11.590946 6161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:18:11.591022 6161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:18:11.593794 5911 master_runner.cc:428] Detected that this master 0ae25ea8906d4eef916298529eaec76d is joining an existing cluster
I20260808 18:18:11.593856 5911 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.18.252:38281
I20260808 18:18:11.603461 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.595622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56042)
0808 18:18:11.595935 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.595946 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.596037 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.596753 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.596758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.597016 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.597170 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.597178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.598041 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.598043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.598799 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.598818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.599522 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.599524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.599584 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.600165 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.600174 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.600455 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.602009 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.602014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.602016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.602205 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.602493 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.602497 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.602498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.602583 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.602946 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.602949 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.603047 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.603162 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.603281 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":136,"threads_started":1}
I20260808 18:18:11.603470 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.596315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.27.18.254:42985)
0808 18:18:11.596448 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.596455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.596471 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:18:11.596510 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.596513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.596557 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.596634 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.597318 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.597916 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.598936 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.599267 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.599336 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.600592 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.600596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.600598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.600621 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.601883 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.602313 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.602317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.602319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.602400 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.602674 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.602678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.602680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.602851 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.602953 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.603217 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.603332 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:18:11.608597 5354 heartbeater.cc:503] Master 127.27.18.254:42985 was elected leader, sending a full tablet report...
I20260808 18:18:11.610608 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 5169
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213090, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.254@KRBTEST.COM for oryx/127.27.18.252@KRBTEST.COM
I20260808 18:18:11.624691 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.596161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.27.18.253:39829)
0808 18:18:11.596289 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.596292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.596832 (+ 540us) server_negotiation.cc:407] Connection header received
0808 18:18:11.596928 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.596931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.596977 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.597065 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.597540 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.598321 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.599419 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.599782 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.599898 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.620683 (+ 20785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.620687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.620689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.620717 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.622215 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623407 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.623411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.623413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.623491 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623781 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.623784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.623786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.623976 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.624040 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.624393 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.624536 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 18:18:11.624943 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.595778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.27.18.252:38281)
0808 18:18:11.596309 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.596313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.603673 (+ 7360us) server_negotiation.cc:407] Connection header received
0808 18:18:11.603761 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.603763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.603801 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.603884 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.604791 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.605420 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.607114 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.607435 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.607497 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.621261 (+ 13764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.621267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.621272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.621300 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.623069 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623596 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.623602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.623606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.623711 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623959 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.623964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.623968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.624172 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.624254 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.624598 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.624728 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466}
I20260808 18:18:11.625170 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.596087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:38986)
0808 18:18:11.596736 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.596748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.596840 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.597095 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.597097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.597324 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.597427 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.597433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.598437 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.598440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.599253 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.599262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.599917 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.599940 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.600045 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.620065 (+ 20020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.620084 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.620544 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.622344 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.622347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.622349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.623281 (+ 932us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.623597 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623599 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.623601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.623691 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.624107 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.624110 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.624197 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.624868 (+ 671us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.625002 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"spinlock_wait_cycles":9856,"thread_start_us":373,"threads_started":1}
I20260808 18:18:11.626044 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.596457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35666)
0808 18:18:11.603569 (+ 7112us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.603585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.603692 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.604389 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.604392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.604572 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.604668 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.604677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.605555 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.605560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.606963 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.606973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.607623 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.607626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.607699 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.620704 (+ 13005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.620723 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.621134 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.623230 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.623240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.623499 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.623793 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.623798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.623803 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.623880 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.624298 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.624303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.624419 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.625649 (+ 1230us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.625800 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7035,"thread_start_us":94,"threads_started":1}
I20260808 18:18:11.629163 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.606916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35674)
0808 18:18:11.607078 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.607092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.607181 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.607473 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.607476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.607646 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.607713 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.607718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.608525 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.608529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.609274 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.609283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.610024 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.610028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.610110 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.610566 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.610577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.617875 (+ 7298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.627560 (+ 9685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.627563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.627565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.627822 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.628167 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.628171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.628173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.628265 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.628701 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.628704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.628796 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.628876 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.629013 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 18:18:11.629891 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.606951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.27.18.252:38281)
0808 18:18:11.607181 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.607189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.607202 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:11.607270 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.607274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.607336 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.607458 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.607836 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.608403 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.609400 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.609890 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.610011 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.618009 (+ 7998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.618013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.618015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.618038 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.626558 (+ 8520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.627935 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.627939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.627940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.628039 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.628388 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.628391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.628393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.628588 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.629551 (+ 963us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.629625 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.629731 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":101,"threads_started":1}
I20260808 18:18:11.629889 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:37223
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:42985
--webserver_port=42625
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:18:11.631623 5069 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.18.252:38281: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0ae25ea8906d4eef916298529eaec76d"
member_type: NON_VOTER
last_known_addr {
host: "127.27.18.252"
port: 38281
}
attrs {
promote: true
}
}
dest_uuid: "5af716cd9591425f8195a786bc9183a8"
cas_config_opid_index: 5
I20260808 18:18:11.632169 5069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } }
I20260808 18:18:11.633196 5903 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0ae25ea8906d4eef916298529eaec76d (127.27.18.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.634696 5903 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.18.252:38281
I20260808 18:18:11.634836 5903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.634959 5903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:11.635854 5911 master.cc:537] Master@127.27.18.252:38281 shutting down...
I20260808 18:18:11.636170 6105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:11.637946 6105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Refusing update from remote peer 5af716cd9591425f8195a786bc9183a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 18:18:11.638672 5902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:18:11.640811 5050 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 -> Peer 0ae25ea8906d4eef916298529eaec76d (127.27.18.252:38281): Couldn't send request to peer 0ae25ea8906d4eef916298529eaec76d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 18:18:11.641215 5911 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:11.641520 5911 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:11.641641 5911 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: stopping tablet replica
I20260808 18:18:11.645131 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.635263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:38988)
0808 18:18:11.635423 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.635438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.640052 (+ 4614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.640400 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.640403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.640422 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:11.641195 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.641204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.642313 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.642317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.643591 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.643602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.644702 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.644706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.644800 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.644859 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.644932 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.644997 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 18:18:11.645541 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.635355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.27.18.253:39829)
0808 18:18:11.635603 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.635607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.635620 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:11.640195 (+ 4575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.640198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.640257 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.640362 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:11.641383 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.642121 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.643741 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.644368 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.644468 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.644620 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.645361 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.645435 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260808 18:18:11.646431 5985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 LEARNER]: Refusing update from remote peer 5af716cd9591425f8195a786bc9183a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 18:18:11.646898 5902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 18:18:11.649014 5911 master.cc:552] Master@127.27.18.252:38281 shutdown complete.
I20260808 18:18:11.649060 5911 master_runner.cc:305] Clearing existing system tablet
I20260808 18:18:11.648842 5985 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0ae25ea8906d4eef916298529eaec76d (127.27.18.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.649296 5911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.649348 5911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.650898 6030 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af716cd9591425f8195a786bc9183a8. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.650998 6030 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:18:11.651075 6030 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: NON_VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: true } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.651134 6030 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: LEARNER
I20260808 18:18:11.651506 5911 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:11.652015 5903 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: attempting to promote NON_VOTER bc0fd7748bd9456bbd69c146e657ee08 to VOTER
I20260808 18:18:11.652340 6189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.652437 5903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } }
I20260808 18:18:11.652606 5911 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.652732 5911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.652773 5911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
uuid: "0ae25ea8906d4eef916298529eaec76d"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "877e08501323784f94bca78948b11539"
server_key_iv: "d09f5760bde0ff77af59a146cf8de3a1"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.653640 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.653390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35684)
0808 18:18:11.653525 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.653582 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 18:18:11.653661 5985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 LEARNER]: Refusing update from remote peer 5af716cd9591425f8195a786bc9183a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 18:18:11.654012 5902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 18:18:11.654412 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.654149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35694)
0808 18:18:11.654316 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.654359 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260808 18:18:11.654564 5050 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 -> Peer 0ae25ea8906d4eef916298529eaec76d (127.27.18.252:38281): Couldn't send request to peer 0ae25ea8906d4eef916298529eaec76d. Status: Network error: Client connection negotiation failed: client connection to 127.27.18.252:38281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:18:11.654793 5911 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:11.655555 5903 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bc0fd7748bd9456bbd69c146e657ee08 (127.27.18.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.656934 5903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.657063 5903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:11.657177 5911 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 18:18:11.657276 5911 log.cc:1196] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 18:18:11.657292 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.657405 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:11.657585 5911 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Deleting consensus metadata
I20260808 18:18:11.657754 5911 master_runner.cc:315] Copying system tablet from 127.27.18.254:42985
I20260808 18:18:11.657994 5985 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bc0fd7748bd9456bbd69c146e657ee08 (127.27.18.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.658802 5911 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: tablet copy: Beginning tablet copy session from remote peer at address 127.27.18.254:42985
I20260808 18:18:11.659248 6185 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.659416 6185 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:11.672380 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.663819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.27.18.254:42985)
0808 18:18:11.663924 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.663929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.664781 (+ 852us) server_negotiation.cc:407] Connection header received
0808 18:18:11.664892 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.664895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.664937 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.665025 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.665546 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.666320 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.667503 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.667814 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.667912 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.669126 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.669130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.669133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.669159 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.670688 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.671148 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.671151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.671153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.671226 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.671562 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.671565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.671567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.671752 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.671820 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.672144 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.672243 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:18:11.672957 6197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.663735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56052)
0808 18:18:11.664694 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.664709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.664807 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.665061 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.665064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.665294 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.665421 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.665428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.666451 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.666455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.667337 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.667347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.667940 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.667944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.668042 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.668570 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.668585 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.668975 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.670808 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.670812 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.670814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.671041 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.671344 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.671347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.671349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.671459 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.671904 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.671906 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.671988 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.672663 (+ 675us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.672791 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":82,"threads_started":1}
I20260808 18:18:11.673597 5097 tablet_copy_service.cc:140] P 5af716cd9591425f8195a786bc9183a8: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0ae25ea8906d4eef916298529eaec76d ({username='oryx', principal='oryx/127.27.18.252@KRBTEST.COM'} at 127.0.0.1:56052)
I20260808 18:18:11.673684 5097 tablet_copy_service.cc:161] P 5af716cd9591425f8195a786bc9183a8: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0ae25ea8906d4eef916298529eaec76d at {username='oryx', principal='oryx/127.27.18.252@KRBTEST.COM'} at 127.0.0.1:56052: session id = 0ae25ea8906d4eef916298529eaec76d-00000000000000000000000000000000
I20260808 18:18:11.674610 5097 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 18:18:11.675789 5911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 18:18:11.677886 5911 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: tablet copy: Starting download of 0 data blocks...
I20260808 18:18:11.678107 5911 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: tablet copy: Starting download of 1 WAL segments...
I20260808 18:18:11.679563 5911 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 18:18:11.681176 5097 tablet_copy_service.cc:342] P 5af716cd9591425f8195a786bc9183a8: Request end of tablet copy session 0ae25ea8906d4eef916298529eaec76d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.18.252@KRBTEST.COM'} at 127.0.0.1:56052
I20260808 18:18:11.681245 5097 tablet_copy_service.cc:433] P 5af716cd9591425f8195a786bc9183a8: ending tablet copy session 0ae25ea8906d4eef916298529eaec76d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0ae25ea8906d4eef916298529eaec76d
I20260808 18:18:11.688063 5911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:11.689575 6198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.692133 6199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.695310 6202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.698869 5911 server_base.cc:1034] running on GCE node
I20260808 18:18:11.699026 5911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:11.699312 5911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:11.700469 5911 hybrid_clock.cc:648] HybridClock initialized: now 1786213091700448 us; error 37 us; skew 500 ppm
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:11.702052 5911 init.cc:377] Logged in from keytab as oryx/127.27.18.252@KRBTEST.COM (short username oryx)
I20260808 18:18:11.702575 5911 webserver.cc:460] Webserver started at http://127.27.18.252:44383/ using document root <none> and password file <none>
I20260808 18:18:11.702764 5911 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.702807 5911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.704175 5911 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.705199 6212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.705469 5911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:11.705546 5911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
uuid: "0ae25ea8906d4eef916298529eaec76d"
format_stamp: "Formatted at 2026-08-08 18:18:11 on dist-test-slave-x4qh"
server_key: "877e08501323784f94bca78948b11539"
server_key_iv: "d09f5760bde0ff77af59a146cf8de3a1"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.705626 5911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.713578 5911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:11.715759 5911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:11.720559 5911 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.252:38281
I20260808 18:18:11.720659 6284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.252:38281 every 8 connection(s)
I20260808 18:18:11.721098 5911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
I20260808 18:18:11.722980 6287 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:18:11.724061 6287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Bootstrap starting.
I20260808 18:18:11.735332 6287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:18:11.735670 6287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: Bootstrap complete.
I20260808 18:18:11.736228 6287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } }
I20260808 18:18:11.736388 6287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0ae25ea8906d4eef916298529eaec76d, State: Initialized, Role: LEARNER
I20260808 18:18:11.736515 6287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } }
I20260808 18:18:11.736887 6290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:11.736984 6287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:11.737089 6290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: This master's current role is: LEARNER
I20260808 18:18:11.740392 6309 catalog_manager.cc:1259] Loaded cluster ID: e585fb888d9840859859ff3058b74515
I20260808 18:18:11.740514 6309 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: loading cluster ID for follower catalog manager: success
I20260808 18:18:11.742604 6309 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: acquiring CA information for follower catalog manager: success
I20260808 18:18:11.744019 6309 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 18:18:11.772099 6180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:11.772320 6180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:11.772399 6180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:11.776463 6180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:11.776512 6180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:11.776605 6180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:11.781435 6180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:37223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=42625
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:11.782338 6180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:11.783237 6180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:11.792410 6324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:11.792431 6320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.792508 6180 server_base.cc:1034] running on GCE node
W20260808 18:18:11.792456 6326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:11.793022 6180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:11.793551 6180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:11.794709 6180 hybrid_clock.cc:648] HybridClock initialized: now 1786213091794695 us; error 25 us; skew 500 ppm
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:11.797421 6180 init.cc:377] Logged in from keytab as oryx/127.27.18.193@KRBTEST.COM (short username oryx)
I20260808 18:18:11.798252 6180 webserver.cc:460] Webserver started at http://127.27.18.193:42625/ using document root <none> and password file <none>
I20260808 18:18:11.798789 6180 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:11.798906 6180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:11.802376 6180 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:18:11.804278 6336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.805538 6180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:11.805662 6180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "46db5413c4c9d8eec104f16b0b40f29a"
server_key_iv: "e858ff6c5558f88989fa18b1630e92d6"
server_key_version: "encryptionkey@0"
I20260808 18:18:11.806198 6180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:11.827148 5555 heartbeater.cc:503] Master 127.27.18.254:42985 was elected leader, sending a full tablet report...
I20260808 18:18:11.829249 6180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:11.832072 6180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:11.832202 6180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:11.832712 6180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:11.833523 6180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:11.833566 6180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.833626 6180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:11.833653 6180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:11.842942 6180 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:37223
I20260808 18:18:11.843043 6508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:37223 every 8 connection(s)
I20260808 18:18:11.843967 6180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:11.850797 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 6180
I20260808 18:18:11.850997 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 5365
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:11.861063 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.847249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:45965 (local address 127.27.18.253:39829)
0808 18:18:11.847400 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.847407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.847665 (+ 258us) server_negotiation.cc:407] Connection header received
0808 18:18:11.847910 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.847914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.847972 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.848029 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.849429 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.850232 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.851591 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.851984 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.852063 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.856296 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.856302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.856308 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.856347 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.858635 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.859233 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.859236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.859238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.859343 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.859876 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.859880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.859882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.860073 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.860186 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.860750 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.860889 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 18:18:11.862036 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.847091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.27.18.193:45965)
0808 18:18:11.847566 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.847581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.847750 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.848268 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.848276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.848710 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.849276 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.849285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.850383 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.850386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.851451 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.851467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.852248 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.852253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.852478 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.853871 (+ 1393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.853896 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.856037 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.858772 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.858780 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.858798 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.859122 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.859510 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.859515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.859520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.859740 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.860188 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.860195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.860501 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.860691 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.861004 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":92,"spinlock_wait_cycles":1408,"thread_start_us":100,"threads_started":1}
I20260808 18:18:11.863461 6514 heartbeater.cc:347] Connected to a master server at 127.27.18.253:39829
I20260808 18:18:11.863669 6514 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:11.864099 6514 heartbeater.cc:511] Master 127.27.18.253:39829 requested a full tablet report, sending...
I20260808 18:18:11.865025 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.846212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:56929 (local address 127.27.18.254:42985)
0808 18:18:11.846387 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.846392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.847626 (+ 1234us) server_negotiation.cc:407] Connection header received
0808 18:18:11.847895 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.847899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.847998 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.848099 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.849406 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.849940 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.856415 (+ 6475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.856654 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.856792 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.859040 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.859047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.859054 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.859094 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.861628 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.862670 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.862677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.862682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.863057 (+ 375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.863449 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.863455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.863460 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.863891 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.864018 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.864597 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.864768 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:18:11.865592 5968 ts_manager.cc:194] Registered new tserver with Master: c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223)
I20260808 18:18:11.865824 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.846075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:56929)
0808 18:18:11.847536 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.847547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.847698 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.848268 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.848274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.848700 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.849255 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.849268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.850084 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.850088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.850998 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.851009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.857025 (+ 6016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.857028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.857105 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.857589 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.857609 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.858869 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.861752 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.861756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.861757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.862006 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.863185 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.863188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.863190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.863289 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.864081 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.864087 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.864311 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.865294 (+ 983us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.865500 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"mutex_wait_us":146,"thread_start_us":134,"threads_started":1}
I20260808 18:18:11.866551 6513 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42985
I20260808 18:18:11.866665 6513 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:11.866907 6513 heartbeater.cc:511] Master 127.27.18.254:42985 requested a full tablet report, sending...
I20260808 18:18:11.870062 5069 ts_manager.cc:194] Re-registered known tserver with Master: c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223)
I20260808 18:18:11.870628 5069 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:56929
Aug 08 18:18:11 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213091, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.252@KRBTEST.COM
I20260808 18:18:11.876766 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:33819
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:42985
--webserver_port=37965
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:18:11.878036 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.845702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:57521 (local address 127.27.18.252:38281)
0808 18:18:11.845983 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:11.845988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:11.846771 (+ 783us) server_negotiation.cc:407] Connection header received
0808 18:18:11.847920 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:11.847923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:11.847972 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:11.848047 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:11.849429 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.850490 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.871792 (+ 21302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.872051 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.872172 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.874640 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:11.874644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:11.874647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:11.874679 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:11.876515 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.876917 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.876922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.876923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.877002 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:11.877267 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:11.877270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:11.877272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:11.877443 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:11.877547 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:11.877786 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 18:18:11.877893 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":91,"threads_started":1}
I20260808 18:18:11.878054 6517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:11.846075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.27.18.193:57521)
0808 18:18:11.846605 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:11.846654 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:11.847696 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:11.848268 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:11.848276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:11.848707 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:11.849277 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.849285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.871010 (+ 21725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.871013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.871664 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:11.871673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:11.872181 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:11.872184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:11.872274 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:11.872751 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:11.872764 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:11.874505 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:11.876626 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.876631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.876633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.876821 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.877094 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:11.877096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:11.877098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:11.877178 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:11.877542 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:11.877545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:11.877634 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 18:18:11.877736 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 18:18:11.877885 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":187,"spinlock_wait_cycles":14080,"thread_start_us":637,"threads_started":1}
I20260808 18:18:11.878772 6512 heartbeater.cc:347] Connected to a master server at 127.27.18.252:38281
I20260808 18:18:11.878873 6512 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:11.879040 6512 heartbeater.cc:511] Master 127.27.18.252:38281 requested a full tablet report, sending...
I20260808 18:18:11.880252 6225 ts_manager.cc:194] Registered new tserver with Master: c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223)
W20260808 18:18:11.988900 6520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:11.989120 6520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:11.989159 6520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:11.992862 6520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:11.992906 6520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:11.993017 6520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:11.997648 6520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:33819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=37965
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:11.998461 6520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:11.999353 6520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:12.008312 6529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:12.008312 6526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:12.008385 6527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:12.008482 6520 server_base.cc:1034] running on GCE node
I20260808 18:18:12.008913 6520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:12.009619 6520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:12.010785 6520 hybrid_clock.cc:648] HybridClock initialized: now 1786213092010758 us; error 36 us; skew 500 ppm
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:12.013432 6520 init.cc:377] Logged in from keytab as oryx/127.27.18.194@KRBTEST.COM (short username oryx)
I20260808 18:18:12.014127 6520 webserver.cc:460] Webserver started at http://127.27.18.194:37965/ using document root <none> and password file <none>
I20260808 18:18:12.014560 6520 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:12.014626 6520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:12.018066 6520 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:12.020027 6537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:12.021138 6520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:12.021245 6520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "d26019add54840f7a3bd87db4626333c"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "cd9c8d6d20e3eb73456afe53e48145a5"
server_key_iv: "0553d30120543e47558bab4f90cb09ac"
server_key_version: "encryptionkey@0"
I20260808 18:18:12.021750 6520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:12.035535 5752 heartbeater.cc:503] Master 127.27.18.254:42985 was elected leader, sending a full tablet report...
I20260808 18:18:12.046571 6520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:12.049152 6520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:12.049315 6520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:12.049758 6520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:12.050487 6520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:12.050529 6520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:12.050563 6520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:12.050583 6520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:12.060313 6520 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33819
I20260808 18:18:12.060357 6707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33819 every 8 connection(s)
I20260808 18:18:12.061182 6520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:12.064633 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 6520
I20260808 18:18:12.064816 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 5560
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.252@KRBTEST.COM
I20260808 18:18:12.080265 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.062701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:45391 (local address 127.27.18.253:39829)
0808 18:18:12.062835 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.062839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.064190 (+ 1351us) server_negotiation.cc:407] Connection header received
0808 18:18:12.067465 (+ 3275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.067470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.067528 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.067580 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.068158 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.068845 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.072176 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.072494 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.072563 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.076126 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.076130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.076132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.076157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.077820 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.078452 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.078457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.078461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.078550 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.078986 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.078990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.078991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.079211 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.079295 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.079987 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.080110 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:12.082340 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.063047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.27.18.194:45391)
0808 18:18:12.064019 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.064090 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.065486 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.067714 (+ 2228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.067719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.067914 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:12.068040 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.068048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.069002 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.069007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.070187 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.070199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.072735 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.072740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.072947 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.073499 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:12.073522 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:12.075944 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:12.077959 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.077966 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.077981 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.078329 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.078669 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.078672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.078675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.078867 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.079383 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:12.079391 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:12.079687 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.080479 (+ 792us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.081341 (+ 862us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"spinlock_wait_cycles":7680,"thread_start_us":641,"threads_started":1}
I20260808 18:18:12.082494 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:42621
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:42985
--webserver_port=37875
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:37049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:18:12.082880 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.064920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:48045 (local address 127.27.18.254:42985)
0808 18:18:12.065050 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.065054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.065069 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:18:12.065747 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.065752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.065835 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.065885 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.070324 (+ 4439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.071005 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.074830 (+ 3825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.075137 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.075202 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.077614 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.077618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.077620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.077647 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.079651 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.081630 (+ 1979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.081633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.081635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.081719 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.082026 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.082029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.082030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.082215 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.082290 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.082631 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.082752 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:18:12.083350 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.064554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:48045)
0808 18:18:12.064946 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.064960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.065640 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.066221 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.066230 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.066685 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:12.067266 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.067286 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.071130 (+ 3844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.071135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.072051 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.072063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.075342 (+ 3279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.075348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.075545 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.076009 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:12.076027 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:12.077455 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:12.080322 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.080325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.080326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.081513 (+ 1187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.081827 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.081829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.081831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.081912 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.082364 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:12.082367 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:12.082451 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.083054 (+ 603us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.083180 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":46,"thread_start_us":96,"threads_started":1}
I20260808 18:18:12.083832 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.063781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:52389 (local address 127.27.18.252:38281)
0808 18:18:12.063911 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.063916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.064191 (+ 275us) server_negotiation.cc:407] Connection header received
0808 18:18:12.073944 (+ 9753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.073947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.074016 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.074120 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.074609 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.075298 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.076551 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.076949 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.077042 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.079563 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.079569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.079575 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.079607 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.081121 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.081736 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.081742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.081747 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.081846 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.082181 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.082184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.082185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.082382 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.082901 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.083466 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.083601 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:18:12.083925 6709 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42985
I20260808 18:18:12.084069 6710 heartbeater.cc:347] Connected to a master server at 127.27.18.253:39829
I20260808 18:18:12.084102 6709 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:12.084159 6710 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:12.084614 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.063673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.27.18.194:52389)
0808 18:18:12.064019 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.064090 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.073806 (+ 9716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.074145 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.074150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.074343 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:12.074478 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.074486 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.075429 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.075435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.076419 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.076432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.077255 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.077260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.077371 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.077855 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:12.077874 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:12.079364 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:12.081244 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.081250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.081254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.081617 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.081967 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.081972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.081977 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.082078 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.083152 (+ 1074us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:12.083157 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:12.083257 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.084296 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.084453 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":39,"spinlock_wait_cycles":14464,"thread_start_us":116,"threads_started":1}
I20260808 18:18:12.084586 6710 heartbeater.cc:511] Master 127.27.18.253:39829 requested a full tablet report, sending...
I20260808 18:18:12.084585 6709 heartbeater.cc:511] Master 127.27.18.254:42985 requested a full tablet report, sending...
I20260808 18:18:12.085762 5069 ts_manager.cc:194] Re-registered known tserver with Master: d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
I20260808 18:18:12.085850 5968 ts_manager.cc:194] Registered new tserver with Master: d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
I20260808 18:18:12.086407 5069 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:48045
I20260808 18:18:12.086961 6708 heartbeater.cc:347] Connected to a master server at 127.27.18.252:38281
I20260808 18:18:12.087020 6708 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:12.087144 6708 heartbeater.cc:511] Master 127.27.18.252:38281 requested a full tablet report, sending...
I20260808 18:18:12.087859 6225 ts_manager.cc:194] Registered new tserver with Master: d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
W20260808 18:18:12.187331 6716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:12.187636 6716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:12.187729 6716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:12.191259 6716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:12.191334 6716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:12.191439 6716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:12.195851 6716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:42621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=37875
--tserver_master_addrs=127.27.18.254:42985,127.27.18.253:39829,127.27.18.252:38281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:12.196751 6716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:12.197639 6716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:12.206367 6727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:12.206449 6724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:12.206422 6723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:12.206777 6716 server_base.cc:1034] running on GCE node
I20260808 18:18:12.207162 6716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:12.207804 6716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:12.208978 6716 hybrid_clock.cc:648] HybridClock initialized: now 1786213092208957 us; error 30 us; skew 500 ppm
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:12.211617 6716 init.cc:377] Logged in from keytab as oryx/127.27.18.195@KRBTEST.COM (short username oryx)
I20260808 18:18:12.212476 6716 webserver.cc:460] Webserver started at http://127.27.18.195:37875/ using document root <none> and password file <none>
I20260808 18:18:12.212953 6716 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:12.213017 6716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:12.216423 6716 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:12.218140 6742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:12.219228 6716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:12.219323 6716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "02e9a23d4880419185b4b55055c86ac4"
format_stamp: "Formatted at 2026-08-08 18:18:10 on dist-test-slave-x4qh"
server_key: "11f533c1f5053dba8f487a3b24c8636b"
server_key_iv: "408cca2dd31431ec1b9aa1682f86857e"
server_key_version: "encryptionkey@0"
I20260808 18:18:12.219815 6716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:12.232708 6716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:12.235123 6716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:12.235256 6716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:12.235787 6716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:12.236528 6716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:12.236572 6716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:12.236605 6716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:12.236624 6716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:12.245664 6716 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42621
I20260808 18:18:12.245685 6912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42621 every 8 connection(s)
I20260808 18:18:12.246534 6716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:12.250001 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 6716
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.252@KRBTEST.COM
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.253@KRBTEST.COM
Aug 08 18:18:12 dist-test-slave-x4qh krb5kdc[5003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213092, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:12.267004 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.248173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:37549 (local address 127.27.18.252:38281)
0808 18:18:12.248318 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.248323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.249333 (+ 1010us) server_negotiation.cc:407] Connection header received
0808 18:18:12.252428 (+ 3095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.252430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.252498 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.252598 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.254248 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.254916 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.257569 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.257914 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.258012 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.261288 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.261294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.261300 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.261331 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.263939 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.265495 (+ 1556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.265500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.265502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.265582 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.265914 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.265917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.265918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.266151 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.266249 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.266713 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.266843 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:18:12.267426 6941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.252109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.27.18.253:39829)
0808 18:18:12.252367 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.252373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.252770 (+ 397us) server_negotiation.cc:407] Connection header received
0808 18:18:12.253000 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.253003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.253057 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.253120 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.254036 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.254792 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.256311 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.256659 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.256731 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.263680 (+ 6949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.263686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.263688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.263720 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.265584 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.266057 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.266060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.266062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.266140 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.266469 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.266475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.266477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.266662 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.266768 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.267104 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.267224 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"spinlock_wait_cycles":2304,"thread_start_us":97,"threads_started":1}
I20260808 18:18:12.267890 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.248620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.27.18.195:37549)
0808 18:18:12.249183 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.249236 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.252073 (+ 2837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.252924 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.252932 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.253413 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:12.254091 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.254110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.255077 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.255097 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.257431 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.257440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.258190 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.258195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.258491 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.259230 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:12.259252 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:12.261106 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:12.264289 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.264297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.264314 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.265381 (+ 1067us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.265689 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.265693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.265695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.265787 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.266292 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:12.266296 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:12.266392 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.266497 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.267308 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":579,"spinlock_wait_cycles":3840,"thread_start_us":116,"threads_started":1}
I20260808 18:18:12.267897 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.249573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.27.18.195:42545)
0808 18:18:12.249841 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.249856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.256517 (+ 6661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.256867 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.256871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.257081 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:12.257217 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.257226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.258279 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.258283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.259195 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.259208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.259990 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.259997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.260122 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.260566 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:12.260580 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:12.262332 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:12.264455 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.264461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.264465 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.264887 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.265205 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.265209 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.265214 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.265399 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.265949 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:12.265959 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:12.266288 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.266500 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.266841 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":1,"spinlock_wait_cycles":10368,"thread_start_us":81,"threads_started":1}
I20260808 18:18:12.268224 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.249669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:42545 (local address 127.27.18.253:39829)
0808 18:18:12.249800 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.249804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.249932 (+ 128us) server_negotiation.cc:407] Connection header received
0808 18:18:12.256641 (+ 6709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.256644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.256689 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.256758 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.257345 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.258140 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.259356 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.259764 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.259869 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.262472 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.262475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.262478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.262521 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.264317 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.265005 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.265008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.265010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.265095 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.265523 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.265527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.265531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.265768 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.265887 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.267981 (+ 2094us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.268090 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:18:12.268317 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.248258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:39525 (local address 127.27.18.254:42985)
0808 18:18:12.248374 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.248377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.249400 (+ 1023us) server_negotiation.cc:407] Connection header received
0808 18:18:12.258915 (+ 9515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.258919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.258969 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.259022 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.260521 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.261229 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.262189 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.262549 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.262636 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.265038 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.265043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.265048 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.265075 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.266683 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.267107 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.267110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.267112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.267187 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.267502 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.267505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.267507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.267685 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.267761 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.268077 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.268171 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:18:12.269011 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.248620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:39525)
0808 18:18:12.249315 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.249330 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.258781 (+ 9451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.260065 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.260068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.260233 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:12.260378 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.260389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.261374 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.261379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.262025 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.262055 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.262806 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.262809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.262900 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.263324 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:12.263344 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:12.264887 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:12.266802 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.266805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.266807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.267001 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.267286 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:12.267288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:12.267290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:12.267400 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:12.267816 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:12.267818 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:12.267897 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.268440 (+ 543us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.268574 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":24,"spinlock_wait_cycles":27776,"thread_start_us":150,"threads_started":1}
I20260808 18:18:12.269527 6913 heartbeater.cc:347] Connected to a master server at 127.27.18.252:38281
I20260808 18:18:12.269755 6913 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:12.269824 6915 heartbeater.cc:347] Connected to a master server at 127.27.18.254:42985
I20260808 18:18:12.269865 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.251897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.27.18.252:38281)
0808 18:18:12.252135 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.252140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.252751 (+ 611us) server_negotiation.cc:407] Connection header received
0808 18:18:12.252924 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.252926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.252964 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.253018 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.253907 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.254723 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.256342 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.256696 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.256767 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.259621 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.259625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.259628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.259652 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.261713 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.264059 (+ 2346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.264063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.264068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.264161 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.268787 (+ 4626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.268791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.268792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.269074 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.269213 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.269562 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.269677 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260808 18:18:12.269853 6916 heartbeater.cc:347] Connected to a master server at 127.27.18.253:39829
I20260808 18:18:12.269888 6915 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:12.269938 6916 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:12.270387 6913 heartbeater.cc:511] Master 127.27.18.252:38281 requested a full tablet report, sending...
I20260808 18:18:12.270967 6916 heartbeater.cc:511] Master 127.27.18.253:39829 requested a full tablet report, sending...
I20260808 18:18:12.271210 6915 heartbeater.cc:511] Master 127.27.18.254:42985 requested a full tablet report, sending...
I20260808 18:18:12.271776 5968 ts_manager.cc:194] Registered new tserver with Master: 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195:42621)
I20260808 18:18:12.271845 5069 ts_manager.cc:194] Re-registered known tserver with Master: 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195:42621)
I20260808 18:18:12.271987 6225 ts_manager.cc:194] Registered new tserver with Master: 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195:42621)
I20260808 18:18:12.272550 5069 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:39525
I20260808 18:18:12.273020 6937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.251672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.27.18.254:42985)
0808 18:18:12.252172 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.252181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.252428 (+ 247us) server_negotiation.cc:407] Connection header received
0808 18:18:12.252661 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.252665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.252741 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.252818 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.253911 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.254754 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.256215 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.256551 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.256622 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.258484 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.258488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.258491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.258516 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.265168 (+ 6652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.270919 (+ 5751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.270922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.270925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.271016 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.271311 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.271314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.271316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.271561 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.271683 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.272026 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.272870 (+ 844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":422,"thread_start_us":80,"threads_started":1}
I20260808 18:18:12.275916 5069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:12.278750 5069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:12.281996 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.275715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35710)
0808 18:18:12.275974 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.275985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.276053 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.279267 (+ 3214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.279269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.279287 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:12.279363 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.279430 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.280291 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.280294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.280975 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.280982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.281666 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.281669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.281747 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.281766 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.281824 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.281888 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":123,"threads_started":1}
I20260808 18:18:12.281987 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.278908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.27.18.252:38281)
0808 18:18:12.279015 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.279019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.279033 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:12.279074 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.279076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.279136 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.279255 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:12.279514 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.280195 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.281088 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.281547 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.281654 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.281785 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.281833 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.281891 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 18:18:12.282465 6246 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 18:18:12.282936 6946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af716cd9591425f8195a786bc9183a8. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:12.283164 6946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: This master's current role is: LEARNER
I20260808 18:18:12.283929 6947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: NON_VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: true } } }
I20260808 18:18:12.284055 6947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:12.286841 6190 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: attempting to promote NON_VOTER 0ae25ea8906d4eef916298529eaec76d to VOTER
I20260808 18:18:12.287176 6190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } }
I20260808 18:18:12.288200 6246 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 LEARNER]: Refusing update from remote peer 5af716cd9591425f8195a786bc9183a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 18:18:12.288589 6948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 18:18:12.289157 5985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Refusing update from remote peer 5af716cd9591425f8195a786bc9183a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 18:18:12.289507 6948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 18:18:12.290306 6947 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0ae25ea8906d4eef916298529eaec76d (127.27.18.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.290786 6246 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0ae25ea8906d4eef916298529eaec76d (127.27.18.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.291776 6947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.291877 6947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:12.291989 6947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.292066 6947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:12.292018 5985 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0ae25ea8906d4eef916298529eaec76d (127.27.18.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.292266 6947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.292368 6947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:12.292677 6946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.292778 6946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:12.293697 6962 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af716cd9591425f8195a786bc9183a8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:12.293834 6962 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:12.299713 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.294399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33819 (local address 127.0.0.1:35054)
0808 18:18:12.294971 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.294981 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.295076 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.295563 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.295565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.295583 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:12.295669 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.295674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.297094 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.297100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.298311 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.298322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.299337 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.299340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.299442 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.299463 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.299535 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.299600 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":83,"threads_started":1}
I20260808 18:18:12.299911 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.294587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.27.18.194:33819)
0808 18:18:12.294938 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.294952 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.295044 (+ 92us) server_negotiation.cc:407] Connection header received
0808 18:18:12.295187 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.295193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.295443 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.295603 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:12.295916 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.296943 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.298446 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.299217 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.299333 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.299487 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.299564 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.299643 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":123,"threads_started":1}
I20260808 18:18:12.300642 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.294196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:37223 (local address 127.0.0.1:49748)
0808 18:18:12.294915 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.294929 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.295003 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.296613 (+ 1610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.296616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.296630 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:12.296714 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.296722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.298085 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.298088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.299150 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.299160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.300277 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.300280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.300368 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.300388 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.300467 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.300527 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":2}
I20260808 18:18:12.300834 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.294370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.27.18.193:37223)
0808 18:18:12.296159 (+ 1789us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.296171 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.296192 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:18:12.296265 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.296270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.296464 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.296851 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:12.296994 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.297976 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.299303 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.300166 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.300257 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.300372 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.300493 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.300571 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1689,"thread_start_us":117,"threads_started":1}
I20260808 18:18:12.300947 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.294848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.27.18.195:42621)
0808 18:18:12.295211 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.295221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.295483 (+ 262us) server_negotiation.cc:407] Connection header received
0808 18:18:12.295561 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.295566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.295743 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.295879 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:12.296766 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.297816 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.299451 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.300154 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.300239 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.300358 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.300685 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.300776 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":99,"threads_started":1}
I20260808 18:18:12.301293 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.294697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42621 (local address 127.0.0.1:44778)
0808 18:18:12.295248 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.295259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.295345 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.296505 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.296508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.296532 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:12.296632 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.296649 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.297966 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.297969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.299309 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.299317 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.300455 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.300459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.300542 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.300563 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.301112 (+ 549us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.301184 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260808 18:18:12.302816 6610 tablet_service.cc:1467] Processing CreateTablet for tablet 484e4ce7f5724345879be4bf7dcf2739 (DEFAULT_TABLE table=test-table [id=fdbb3c57570a4645a87914670fac9c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:12.303179 6404 tablet_service.cc:1467] Processing CreateTablet for tablet 484e4ce7f5724345879be4bf7dcf2739 (DEFAULT_TABLE table=test-table [id=fdbb3c57570a4645a87914670fac9c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:12.303941 6610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484e4ce7f5724345879be4bf7dcf2739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:12.304196 6404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484e4ce7f5724345879be4bf7dcf2739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:12.303898 6814 tablet_service.cc:1467] Processing CreateTablet for tablet 484e4ce7f5724345879be4bf7dcf2739 (DEFAULT_TABLE table=test-table [id=fdbb3c57570a4645a87914670fac9c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:12.304840 6814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484e4ce7f5724345879be4bf7dcf2739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:12.311985 6976 tablet_bootstrap.cc:492] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Bootstrap starting.
I20260808 18:18:12.313071 6979 tablet_bootstrap.cc:492] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Bootstrap starting.
I20260808 18:18:12.313961 6976 tablet_bootstrap.cc:654] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:12.314642 6979 tablet_bootstrap.cc:654] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:12.314635 6976 log.cc:824] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:12.315466 6979 log.cc:824] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:12.315666 6975 tablet_bootstrap.cc:492] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Bootstrap starting.
I20260808 18:18:12.316576 6976 tablet_bootstrap.cc:492] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: No bootstrap required, opened a new log
I20260808 18:18:12.316756 6976 ts_tablet_manager.cc:1397] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:18:12.317245 6979 tablet_bootstrap.cc:492] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: No bootstrap required, opened a new log
I20260808 18:18:12.317413 6979 ts_tablet_manager.cc:1397] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 18:18:12.317572 6975 tablet_bootstrap.cc:654] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:12.318382 6975 log.cc:824] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:12.320415 6975 tablet_bootstrap.cc:492] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: No bootstrap required, opened a new log
I20260808 18:18:12.320348 6976 raft_consensus.cc:348] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.320568 6976 raft_consensus.cc:374] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:12.320580 6975 ts_tablet_manager.cc:1397] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:18:12.320623 6976 raft_consensus.cc:729] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c54b5b6fa5ed47bbaf7593e1991a10f5, State: Initialized, Role: FOLLOWER
I20260808 18:18:12.321022 6976 consensus_queue.cc:260] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.321609 6513 heartbeater.cc:503] Master 127.27.18.254:42985 was elected leader, sending a full tablet report...
I20260808 18:18:12.321543 6979 raft_consensus.cc:348] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.321759 6979 raft_consensus.cc:374] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:12.321811 6979 raft_consensus.cc:729] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e9a23d4880419185b4b55055c86ac4, State: Initialized, Role: FOLLOWER
I20260808 18:18:12.322190 6979 consensus_queue.cc:260] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.323490 6915 heartbeater.cc:503] Master 127.27.18.254:42985 was elected leader, sending a full tablet report...
I20260808 18:18:12.324198 6979 ts_tablet_manager.cc:1428] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260808 18:18:12.327268 6976 ts_tablet_manager.cc:1428] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260808 18:18:12.328101 6975 raft_consensus.cc:348] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.328320 6975 raft_consensus.cc:374] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:12.328370 6975 raft_consensus.cc:729] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26019add54840f7a3bd87db4626333c, State: Initialized, Role: FOLLOWER
I20260808 18:18:12.328783 6975 consensus_queue.cc:260] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.329422 6709 heartbeater.cc:503] Master 127.27.18.254:42985 was elected leader, sending a full tablet report...
I20260808 18:18:12.329631 6975 ts_tablet_manager.cc:1428] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Time spent starting tablet: real 0.009s user 0.001s sys 0.005s
I20260808 18:18:12.341150 6986 raft_consensus.cc:482] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:12.341311 6986 raft_consensus.cc:504] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.342214 6986 leader_election.cc:288] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223), d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
W20260808 18:18:12.345405 6515 tablet.cc:2369] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:12.346344 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.342520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:44243 (local address 127.27.18.193:37223)
0808 18:18:12.342629 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.342634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.342646 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:12.342791 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.342807 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.342850 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.342916 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:12.343287 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.343910 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.345147 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.345743 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.345848 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.345897 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.346139 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.346214 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:18:12.346359 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.342409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:37223 (local address 127.27.18.195:44243)
0808 18:18:12.342548 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.342565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.342650 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.343030 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.343034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.343052 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:12.343164 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.343170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.344019 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.344024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.345023 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.345033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.345911 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.345914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.346013 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.346038 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.346114 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.346185 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 18:18:12.346869 6437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484e4ce7f5724345879be4bf7dcf2739" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" is_pre_election: true
I20260808 18:18:12.347051 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.342638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33819 (local address 127.27.18.195:43929)
0808 18:18:12.342748 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:12.342761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:12.342830 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:12.343410 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:12.343413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:12.343431 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:12.343530 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.343536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.344473 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.344476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.345508 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:12.345519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.346657 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.346661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:12.346764 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.346787 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:18:12.346874 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:18:12.346943 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 18:18:12.347059 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.343014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:43929 (local address 127.27.18.194:33819)
0808 18:18:12.343109 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.343113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.343126 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:12.343165 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.343168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.343213 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.343279 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:12.343642 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.344353 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.345706 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.346268 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.346409 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.346460 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.346892 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.346962 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:18:12.347149 6437 raft_consensus.cc:2454] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 0.
I20260808 18:18:12.347615 6743 leader_election.cc:302] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02e9a23d4880419185b4b55055c86ac4, c54b5b6fa5ed47bbaf7593e1991a10f5; no voters:
I20260808 18:18:12.347615 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484e4ce7f5724345879be4bf7dcf2739" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26019add54840f7a3bd87db4626333c" is_pre_election: true
I20260808 18:18:12.347834 6621 raft_consensus.cc:2454] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 0.
I20260808 18:18:12.347868 6986 raft_consensus.cc:2781] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:12.347975 6986 raft_consensus.cc:482] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:12.348040 6986 raft_consensus.cc:3037] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:12.348920 6986 raft_consensus.cc:504] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.349236 6986 leader_election.cc:288] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223), d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
I20260808 18:18:12.349574 6437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484e4ce7f5724345879be4bf7dcf2739" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5"
I20260808 18:18:12.349632 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484e4ce7f5724345879be4bf7dcf2739" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26019add54840f7a3bd87db4626333c"
I20260808 18:18:12.349721 6621 raft_consensus.cc:3037] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:12.349713 6437 raft_consensus.cc:3037] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:12.350909 6621 raft_consensus.cc:2454] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 1.
I20260808 18:18:12.350909 6437 raft_consensus.cc:2454] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 1.
I20260808 18:18:12.351166 6743 leader_election.cc:302] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02e9a23d4880419185b4b55055c86ac4, d26019add54840f7a3bd87db4626333c; no voters:
I20260808 18:18:12.351341 6986 raft_consensus.cc:2781] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:12.351563 6986 raft_consensus.cc:686] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 02e9a23d4880419185b4b55055c86ac4, State: Running, Role: LEADER
I20260808 18:18:12.351840 6986 consensus_queue.cc:237] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } }
I20260808 18:18:12.354622 5069 catalog_manager.cc:5495] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "02e9a23d4880419185b4b55055c86ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } health_report { overall_health: HEALTHY } } }
I20260808 18:18:12.378466 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.375266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.27.18.195:42621)
0808 18:18:12.375418 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.375424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.375439 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:18:12.375607 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.375610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.375660 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.375757 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:12.376005 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.376463 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.377220 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.377425 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.377519 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.377795 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:12.378119 (+ 324us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:12.378236 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.378296 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.378351 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:18:12.383287 6621 raft_consensus.cc:1261] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Refusing update from remote peer 02e9a23d4880419185b4b55055c86ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:12.383302 6437 raft_consensus.cc:1261] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Refusing update from remote peer 02e9a23d4880419185b4b55055c86ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:12.383947 6986 consensus_queue.cc:1035] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:12.384145 6988 consensus_queue.cc:1035] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:12.392041 6993 mvcc.cc:205] Tried to move back new op lower bound from 7316328826396270592 to 7316328826275340288. Current Snapshot: MvccSnapshot[applied={T|T < 7316328826396270592}]
I20260808 18:18:12.392427 6998 mvcc.cc:205] Tried to move back new op lower bound from 7316328826396270592 to 7316328826275340288. Current Snapshot: MvccSnapshot[applied={T|T < 7316328826396270592}]
I20260808 18:18:12.403998 5069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:12.404138 5069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:12.408654 5069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5af716cd9591425f8195a786bc9183a8: Sending DeleteTablet for 3 replicas of tablet 484e4ce7f5724345879be4bf7dcf2739
I20260808 18:18:12.409413 6404 tablet_service.cc:1514] Processing DeleteTablet for tablet 484e4ce7f5724345879be4bf7dcf2739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:12 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:49748
I20260808 18:18:12.410087 6610 tablet_service.cc:1514] Processing DeleteTablet for tablet 484e4ce7f5724345879be4bf7dcf2739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:12 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:35054
I20260808 18:18:12.410109 6814 tablet_service.cc:1514] Processing DeleteTablet for tablet 484e4ce7f5724345879be4bf7dcf2739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:12 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:44778
I20260808 18:18:12.410769 7015 tablet_replica.cc:330] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: stopping tablet replica
I20260808 18:18:12.411254 7010 tablet_replica.cc:330] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: stopping tablet replica
I20260808 18:18:12.411582 7010 raft_consensus.cc:2229] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:12.411926 7010 raft_consensus.cc:2258] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:12.413262 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 5020
I20260808 18:18:12.413548 7010 ts_tablet_manager.cc:1905] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:12.416761 7010 ts_tablet_manager.cc:1918] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:12.416846 7010 log.cc:1196] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/484e4ce7f5724345879be4bf7dcf2739
I20260808 18:18:12.417181 7010 ts_tablet_manager.cc:1939] T 484e4ce7f5724345879be4bf7dcf2739 P c54b5b6fa5ed47bbaf7593e1991a10f5: Deleting consensus metadata
I20260808 18:18:12.418427 7016 tablet_replica.cc:330] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: stopping tablet replica
I20260808 18:18:12.418727 7016 raft_consensus.cc:2229] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:12.419131 7016 raft_consensus.cc:2258] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:12.422161 7016 ts_tablet_manager.cc:1905] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:12.423605 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.412206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.27.18.253:39829)
0808 18:18:12.412320 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.412325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.412338 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:12.412375 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.412379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.412422 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.412493 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.413535 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.414213 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.417757 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.418066 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.418146 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.420111 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.420115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.420118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.420142 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.421751 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.422247 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.422251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.422253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.422347 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.422669 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.422672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.422674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.422870 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.422967 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.423301 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.423467 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:18:12.425107 7016 ts_tablet_manager.cc:1918] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:12.425195 7016 log.cc:1196] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/484e4ce7f5724345879be4bf7dcf2739
I20260808 18:18:12.425562 7016 ts_tablet_manager.cc:1939] T 484e4ce7f5724345879be4bf7dcf2739 P 02e9a23d4880419185b4b55055c86ac4: Deleting consensus metadata
I20260808 18:18:12.411118 7015 raft_consensus.cc:2229] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:12.427994 7015 raft_consensus.cc:2258] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 18:18:12.431913 6746 connection.cc:535] server connection from 127.0.0.1:44778 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 18:18:12.432498 6338 connection.cc:535] server connection from 127.0.0.1:49748 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:18:12.432842 7015 ts_tablet_manager.cc:1905] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:12.433118 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.412792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.27.18.252:38281)
0808 18:18:12.412906 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.412911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.424933 (+ 12022us) server_negotiation.cc:407] Connection header received
0808 18:18:12.425143 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.425146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.425208 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.425267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.426165 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.426803 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.427607 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.427891 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.427962 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.429801 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.429812 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.429815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.429844 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.431196 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.431714 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.431720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.431723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.431818 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.432160 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.432164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.432166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.432399 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.432511 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.432848 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.432983 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:18:12.435731 7015 ts_tablet_manager.cc:1918] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:12.435846 7015 log.cc:1196] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/484e4ce7f5724345879be4bf7dcf2739
I20260808 18:18:12.436208 7015 ts_tablet_manager.cc:1939] T 484e4ce7f5724345879be4bf7dcf2739 P d26019add54840f7a3bd87db4626333c: Deleting consensus metadata
W20260808 18:18:12.437274 6541 connection.cc:441] server connection from 127.0.0.1:35054 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35054 (request call id 1) could send its response
I20260808 18:18:12.445534 6145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.437101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.27.18.253:39829)
0808 18:18:12.437228 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.437233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.437246 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:12.437282 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.437284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.437329 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.437434 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.438319 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.438782 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.439712 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.439934 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.440031 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.441435 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.441439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.441441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.441465 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.443034 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.443901 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.443904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.443905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.443978 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.444257 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.444261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.444262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.444490 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.444614 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.444880 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.445113 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:18:12.447860 6940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:12.437617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.27.18.252:38281)
0808 18:18:12.437708 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:12.437711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:12.440777 (+ 3066us) server_negotiation.cc:407] Connection header received
0808 18:18:12.440871 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:12.440877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:12.440930 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:12.441020 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:12.441788 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.442389 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.443150 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:12.443349 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:12.443462 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:12.444932 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:12.444936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:12.444938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:12.444962 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:12.446285 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.446696 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.446700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.446702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.446777 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:12.447056 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:12.447059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:12.447061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:12.447206 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:12.447350 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:12.447600 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 18:18:12.447722 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 18:18:13.395583 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.395052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:43769)
0808 18:18:13.395323 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.395451 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
W20260808 18:18:13.396039 6513 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:42985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.398623 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.397847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:52059)
0808 18:18:13.398229 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.398531 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":67,"threads_started":1}
W20260808 18:18:13.398936 6915 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:42985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.399044 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.398653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:56225)
0808 18:18:13.398895 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.398966 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":51,"threads_started":1}
W20260808 18:18:13.399284 6709 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:42985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.909044 7030 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5af716cd9591425f8195a786bc9183a8)
I20260808 18:18:13.909245 7030 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } }
I20260808 18:18:13.910277 7031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5af716cd9591425f8195a786bc9183a8)
I20260808 18:18:13.910405 7030 leader_election.cc:288] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985), 0ae25ea8906d4eef916298529eaec76d (127.27.18.252:38281)
I20260808 18:18:13.910475 7031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } }
I20260808 18:18:13.910902 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.910467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56494)
0808 18:18:13.910694 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.910792 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
W20260808 18:18:13.911125 5941 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.911464 7031 leader_election.cc:288] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985), bc0fd7748bd9456bbd69c146e657ee08 (127.27.18.253:39829)
I20260808 18:18:13.912245 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.911895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56522)
0808 18:18:13.912110 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.912158 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":49,"threads_started":1}
I20260808 18:18:13.912299 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.911608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56506)
0808 18:18:13.912161 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.912217 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":78,"threads_started":1}
W20260808 18:18:13.912360 5941 leader_election.cc:334] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985): Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
W20260808 18:18:13.912424 6213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.913163 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.912952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56536)
0808 18:18:13.913053 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.913107 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 18:18:13.913295 6213 leader_election.cc:334] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985): Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.914868 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.910858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.27.18.252:38281)
0808 18:18:13.911154 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:13.911159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:13.911219 (+ 60us) server_negotiation.cc:407] Connection header received
0808 18:18:13.911364 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:13.911384 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:13.911433 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:13.911521 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:13.911855 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.912575 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.913912 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.914332 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.914459 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.914528 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:13.914702 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 18:18:13.914773 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
I20260808 18:18:13.914865 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.911046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:38281 (local address 127.0.0.1:35740)
0808 18:18:13.911138 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.911152 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:13.911257 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:13.911547 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:13.911550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:13.911572 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:13.911678 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.911685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.912679 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.912685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.913792 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.913802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.914461 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.914464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.914583 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.914600 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:18:13.914684 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 18:18:13.914750 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 18:18:13.915343 6246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc0fd7748bd9456bbd69c146e657ee08" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ae25ea8906d4eef916298529eaec76d" is_pre_election: true
I20260808 18:18:13.915592 6246 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0fd7748bd9456bbd69c146e657ee08 in term 1.
I20260808 18:18:13.915962 5944 leader_election.cc:302] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae25ea8906d4eef916298529eaec76d, bc0fd7748bd9456bbd69c146e657ee08; no voters: 5af716cd9591425f8195a786bc9183a8
I20260808 18:18:13.916194 7030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 18:18:13.916263 7030 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5af716cd9591425f8195a786bc9183a8)
I20260808 18:18:13.916338 7030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:18:13.916564 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.911944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:39829 (local address 127.0.0.1:39034)
0808 18:18:13.913020 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.913029 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:13.913124 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:13.913356 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:13.913359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:13.913378 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:13.913470 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.913475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.914145 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.914148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.915014 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.915028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.916200 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.916204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.916294 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.916346 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 18:18:13.916424 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:18:13.916483 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":57,"threads_started":1}
I20260808 18:18:13.916877 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.911964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39034 (local address 127.27.18.253:39829)
0808 18:18:13.912867 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:13.912876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:13.913093 (+ 217us) server_negotiation.cc:407] Connection header received
0808 18:18:13.913196 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:13.913199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:13.913245 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:13.913332 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:13.913566 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.914039 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.915157 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.915799 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.915913 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.915962 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:13.916736 (+ 774us) server_negotiation.cc:299] Negotiation successful
0808 18:18:13.916800 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":100,"threads_started":1}
I20260808 18:18:13.917227 5985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ae25ea8906d4eef916298529eaec76d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bc0fd7748bd9456bbd69c146e657ee08" is_pre_election: true
I20260808 18:18:13.917635 7030 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } }
I20260808 18:18:13.917881 5985 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ae25ea8906d4eef916298529eaec76d in current term 2: Already voted for candidate bc0fd7748bd9456bbd69c146e657ee08 in this term.
I20260808 18:18:13.918004 7030 leader_election.cc:288] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [CANDIDATE]: Term 2 election: Requested vote from peers 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985), 0ae25ea8906d4eef916298529eaec76d (127.27.18.252:38281)
I20260808 18:18:13.918285 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.918110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56550)
0808 18:18:13.918180 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.918225 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 18:18:13.918274 6213 leader_election.cc:302] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae25ea8906d4eef916298529eaec76d; no voters: 5af716cd9591425f8195a786bc9183a8, bc0fd7748bd9456bbd69c146e657ee08
I20260808 18:18:13.918404 6246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc0fd7748bd9456bbd69c146e657ee08" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ae25ea8906d4eef916298529eaec76d"
I20260808 18:18:13.918511 6246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:18:13.918924 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.918719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56564)
0808 18:18:13.918819 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.918879 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:18:13.919049 5941 leader_election.cc:334] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985): Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
I20260808 18:18:13.919688 6246 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0fd7748bd9456bbd69c146e657ee08 in term 2.
I20260808 18:18:13.919797 7031 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:18:13.920017 5944 leader_election.cc:302] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ae25ea8906d4eef916298529eaec76d, bc0fd7748bd9456bbd69c146e657ee08; no voters: 5af716cd9591425f8195a786bc9183a8
I20260808 18:18:13.920229 7030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:18:13.920449 7030 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [term 2 LEADER]: Becoming Leader. State: Replica: bc0fd7748bd9456bbd69c146e657ee08, State: Running, Role: LEADER
I20260808 18:18:13.920699 7030 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } }
I20260808 18:18:13.921716 7041 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc0fd7748bd9456bbd69c146e657ee08. Latest consensus state: current_term: 2 leader_uuid: "bc0fd7748bd9456bbd69c146e657ee08" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:13.921808 7041 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:13.922089 7046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:13.924494 7046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:13.924849 7046 catalog_manager.cc:1259] Loaded cluster ID: e585fb888d9840859859ff3058b74515
I20260808 18:18:13.924892 7046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:13.925413 7046 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:13.925772 7046 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Loaded TSK: 0
I20260808 18:18:13.926354 7046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:18:13.946650 5967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:13.949390 5967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:13.951853 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.951627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56576)
0808 18:18:13.951733 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.951779 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 18:18:13.951855 6246 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [term 2 FOLLOWER]: Refusing update from remote peer bc0fd7748bd9456bbd69c146e657ee08: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 18:18:13.952524 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.952302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.0.0.1:56580)
0808 18:18:13.952423 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.952472 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:18:13.952459 7041 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 18:18:13.952677 5941 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 -> Peer 5af716cd9591425f8195a786bc9183a8 (127.27.18.254:42985): Couldn't send request to peer 5af716cd9591425f8195a786bc9183a8. Status: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:18:13.954007 7031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc0fd7748bd9456bbd69c146e657ee08. Latest consensus state: current_term: 2 leader_uuid: "bc0fd7748bd9456bbd69c146e657ee08" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:13.954109 7031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:13.955057 7030 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bc0fd7748bd9456bbd69c146e657ee08" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:13.955169 7030 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:13.955473 7050 mvcc.cc:205] Tried to move back new op lower bound from 7316328832823250944 to 7316328832701464576. Current Snapshot: MvccSnapshot[applied={T|T < 7316328832823250944}]
I20260808 18:18:13.955605 7031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bc0fd7748bd9456bbd69c146e657ee08" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5af716cd9591425f8195a786bc9183a8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42985 } } peers { permanent_uuid: "bc0fd7748bd9456bbd69c146e657ee08" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 39829 } attrs { promote: false } } peers { permanent_uuid: "0ae25ea8906d4eef916298529eaec76d" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 38281 } attrs { promote: false } } }
I20260808 18:18:13.955689 7031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae25ea8906d4eef916298529eaec76d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:18:13.965492 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.960856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42621 (local address 127.0.0.1:44804)
0808 18:18:13.961162 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.961176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:13.961261 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:13.962376 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:13.962381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:13.962411 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:13.962480 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.962484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.963402 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.963408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.964062 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.964069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.965054 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.965057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.965179 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.965215 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:18:13.965289 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:18:13.965366 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260808 18:18:13.965529 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.961018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:37223 (local address 127.0.0.1:49754)
0808 18:18:13.961405 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.961422 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:13.961503 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:13.962161 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:13.962165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:13.962183 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:13.962272 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.962280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.963348 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.963351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.964375 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.964384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.965145 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.965148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.965252 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.965294 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 18:18:13.965345 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 18:18:13.965406 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317}
I20260808 18:18:13.965663 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.960958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.27.18.195:42621)
0808 18:18:13.962149 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:13.962153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:13.962165 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:13.962213 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:13.962215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:13.962260 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:13.962360 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:13.962579 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.963244 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.964175 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.964944 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.965052 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.965147 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:13.965324 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 18:18:13.965400 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":63,"threads_started":1}
I20260808 18:18:13.965814 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.961349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33819 (local address 127.0.0.1:35070)
0808 18:18:13.961580 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.961607 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:13.961696 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:13.962206 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:13.962210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:13.962228 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:13.962342 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.962351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.963139 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.963144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.964216 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:13.964225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.965297 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.965300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:13.965375 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.965395 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:18:13.965440 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 18:18:13.965686 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260808 18:18:13.966003 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.961665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.27.18.193:37223)
0808 18:18:13.961901 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:13.961906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:13.961921 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:18:13.961976 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:13.961980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:13.962031 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:13.962399 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:13.962500 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.963246 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.964472 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.965006 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.965692 (+ 686us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.965768 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:13.965815 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 18:18:13.965887 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260808 18:18:13.966090 6814 tablet_service.cc:1467] Processing CreateTablet for tablet e721bc2d9eb14ccea6c458da56909cc7 (DEFAULT_TABLE table=test-table [id=712edabaebb64c1e9c5d3805e12edc83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:13.966367 6814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e721bc2d9eb14ccea6c458da56909cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:13.966643 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.961705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.27.18.194:33819)
0808 18:18:13.961942 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:13.961946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:13.961958 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:13.962019 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:13.962023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:13.962077 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:13.962462 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:13.962575 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.963039 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.964595 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:13.965163 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:13.966352 (+ 1189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:13.966400 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:13.966441 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 18:18:13.966524 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260808 18:18:13.967111 6610 tablet_service.cc:1467] Processing CreateTablet for tablet e721bc2d9eb14ccea6c458da56909cc7 (DEFAULT_TABLE table=test-table [id=712edabaebb64c1e9c5d3805e12edc83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:13.967361 6610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e721bc2d9eb14ccea6c458da56909cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:13.967872 6404 tablet_service.cc:1467] Processing CreateTablet for tablet e721bc2d9eb14ccea6c458da56909cc7 (DEFAULT_TABLE table=test-table [id=712edabaebb64c1e9c5d3805e12edc83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:13.968125 6404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e721bc2d9eb14ccea6c458da56909cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:13.968948 7061 tablet_bootstrap.cc:492] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Bootstrap starting.
I20260808 18:18:13.969905 7061 tablet_bootstrap.cc:654] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:13.969991 7063 tablet_bootstrap.cc:492] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Bootstrap starting.
I20260808 18:18:13.970772 7063 tablet_bootstrap.cc:654] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:13.970777 7068 tablet_bootstrap.cc:492] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5: Bootstrap starting.
I20260808 18:18:13.970955 7061 tablet_bootstrap.cc:492] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: No bootstrap required, opened a new log
I20260808 18:18:13.971040 7061 ts_tablet_manager.cc:1397] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:13.971443 7061 raft_consensus.cc:348] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:13.971544 7068 tablet_bootstrap.cc:654] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:13.971572 7061 raft_consensus.cc:374] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:13.971609 7061 raft_consensus.cc:729] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e9a23d4880419185b4b55055c86ac4, State: Initialized, Role: FOLLOWER
I20260808 18:18:13.971733 7061 consensus_queue.cc:260] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:13.971962 7063 tablet_bootstrap.cc:492] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: No bootstrap required, opened a new log
I20260808 18:18:13.972054 7063 ts_tablet_manager.cc:1397] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:13.972110 7061 ts_tablet_manager.cc:1428] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:13.972797 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.972430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:48467)
0808 18:18:13.972662 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.972716 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":117,"threads_started":1}
I20260808 18:18:13.974928 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.974710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:36157)
0808 18:18:13.974837 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.974879 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 18:18:13.975199 7063 raft_consensus.cc:348] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:13.975315 7063 raft_consensus.cc:374] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:13.975360 7063 raft_consensus.cc:729] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26019add54840f7a3bd87db4626333c, State: Initialized, Role: FOLLOWER
I20260808 18:18:13.975509 7063 consensus_queue.cc:260] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:13.975790 7063 ts_tablet_manager.cc:1428] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260808 18:18:13.976485 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.976091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:45527)
0808 18:18:13.976319 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.976382 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":99,"threads_started":1}
I20260808 18:18:13.976629 7068 tablet_bootstrap.cc:492] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5: No bootstrap required, opened a new log
I20260808 18:18:13.976732 7068 ts_tablet_manager.cc:1397] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 18:18:13.977149 7068 raft_consensus.cc:348] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:13.977262 7068 raft_consensus.cc:374] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:13.977290 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.976986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:50597)
0808 18:18:13.977181 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.977228 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260808 18:18:13.977310 7068 raft_consensus.cc:729] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c54b5b6fa5ed47bbaf7593e1991a10f5, State: Initialized, Role: FOLLOWER
I20260808 18:18:13.977443 7068 consensus_queue.cc:260] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:13.977833 7068 ts_tablet_manager.cc:1428] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:13.978439 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.978041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:52255)
0808 18:18:13.978284 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.978334 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260808 18:18:13.979045 6710 heartbeater.cc:503] Master 127.27.18.253:39829 was elected leader, sending a full tablet report...
I20260808 18:18:13.979298 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:13.978834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:46065)
0808 18:18:13.979181 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:13.979227 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":98,"threads_started":1}
I20260808 18:18:13.980316 6514 heartbeater.cc:503] Master 127.27.18.253:39829 was elected leader, sending a full tablet report...
W20260808 18:18:13.998776 6917 tablet.cc:2369] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:18:14.064088 6711 tablet.cc:2369] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:14.111647 7074 raft_consensus.cc:482] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:14.111771 7074 raft_consensus.cc:504] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:14.112179 7074 leader_election.cc:288] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223), d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
I20260808 18:18:14.112563 6437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e721bc2d9eb14ccea6c458da56909cc7" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" is_pre_election: true
I20260808 18:18:14.112610 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e721bc2d9eb14ccea6c458da56909cc7" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26019add54840f7a3bd87db4626333c" is_pre_election: true
I20260808 18:18:14.112722 6437 raft_consensus.cc:2454] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 0.
I20260808 18:18:14.112735 6621 raft_consensus.cc:2454] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 0.
I20260808 18:18:14.113023 6743 leader_election.cc:302] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02e9a23d4880419185b4b55055c86ac4, d26019add54840f7a3bd87db4626333c; no voters:
I20260808 18:18:14.113188 7074 raft_consensus.cc:2781] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:14.113245 7074 raft_consensus.cc:482] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:14.113293 7074 raft_consensus.cc:3037] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:14.114295 7074 raft_consensus.cc:504] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:14.114599 7074 leader_election.cc:288] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 election: Requested vote from peers c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223), d26019add54840f7a3bd87db4626333c (127.27.18.194:33819)
I20260808 18:18:14.114960 6437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e721bc2d9eb14ccea6c458da56909cc7" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5"
I20260808 18:18:14.114988 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e721bc2d9eb14ccea6c458da56909cc7" candidate_uuid: "02e9a23d4880419185b4b55055c86ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26019add54840f7a3bd87db4626333c"
I20260808 18:18:14.115083 6437 raft_consensus.cc:3037] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:14.115097 6621 raft_consensus.cc:3037] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:14.116228 6437 raft_consensus.cc:2454] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 1.
I20260808 18:18:14.116228 6621 raft_consensus.cc:2454] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02e9a23d4880419185b4b55055c86ac4 in term 1.
I20260808 18:18:14.116536 6743 leader_election.cc:302] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02e9a23d4880419185b4b55055c86ac4, c54b5b6fa5ed47bbaf7593e1991a10f5; no voters:
I20260808 18:18:14.116691 7074 raft_consensus.cc:2781] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:14.116902 7074 raft_consensus.cc:686] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 02e9a23d4880419185b4b55055c86ac4, State: Running, Role: LEADER
I20260808 18:18:14.117050 7074 consensus_queue.cc:237] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } }
I20260808 18:18:14.117327 6916 heartbeater.cc:503] Master 127.27.18.253:39829 was elected leader, sending a full tablet report...
I20260808 18:18:14.117622 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.117346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:54545)
0808 18:18:14.117487 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.117547 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 18:18:14.118875 5967 catalog_manager.cc:5495] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "02e9a23d4880419185b4b55055c86ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02e9a23d4880419185b4b55055c86ac4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 } health_report { overall_health: UNKNOWN } } }
I20260808 18:18:14.153457 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.150786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.27.18.195:42621)
0808 18:18:14.150914 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:14.150918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:14.150932 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:14.150975 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:14.150977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:14.151017 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:14.151078 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:14.151323 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:14.151827 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:14.152580 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:14.152803 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:14.152877 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:14.153101 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:14.153165 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:14.153215 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:14.153312 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 18:18:14.153358 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:14.155493 6621 raft_consensus.cc:1261] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Refusing update from remote peer 02e9a23d4880419185b4b55055c86ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:14.155565 6437 raft_consensus.cc:1261] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Refusing update from remote peer 02e9a23d4880419185b4b55055c86ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:14.155920 7074 consensus_queue.cc:1035] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26019add54840f7a3bd87db4626333c" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:14.156062 7085 consensus_queue.cc:1035] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c54b5b6fa5ed47bbaf7593e1991a10f5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:14.157934 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.157619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:37283)
0808 18:18:14.157778 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.157838 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 18:18:14.158205 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.157969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:53121)
0808 18:18:14.158088 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.158143 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 18:18:14.161422 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.161044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:48783)
0808 18:18:14.161289 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.161352 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260808 18:18:14.162901 7092 mvcc.cc:205] Tried to move back new op lower bound from 7316328833656635392 to 7316328833504739328. Current Snapshot: MvccSnapshot[applied={T|T < 7316328833656635392 or (T in {7316328833656635392})}]
I20260808 18:18:14.163197 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.162954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.195:34883)
0808 18:18:14.163095 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.163140 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 18:18:14.163220 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.162931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.193:34259)
0808 18:18:14.163128 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.163175 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 18:18:14.163889 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.163691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:42985 (local address 127.27.18.194:58955)
0808 18:18:14.163778 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.163840 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:42985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 18:18:14.168254 5967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:14.168385 5967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:14.171672 5967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc0fd7748bd9456bbd69c146e657ee08: Sending DeleteTablet for 3 replicas of tablet e721bc2d9eb14ccea6c458da56909cc7
I20260808 18:18:14.172217 6814 tablet_service.cc:1514] Processing DeleteTablet for tablet e721bc2d9eb14ccea6c458da56909cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:14 UTC) from {username='oryx', principal='oryx/127.27.18.253@KRBTEST.COM'} at 127.0.0.1:44804
I20260808 18:18:14.172372 6610 tablet_service.cc:1514] Processing DeleteTablet for tablet e721bc2d9eb14ccea6c458da56909cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:14 UTC) from {username='oryx', principal='oryx/127.27.18.253@KRBTEST.COM'} at 127.0.0.1:35070
I20260808 18:18:14.172508 7106 tablet_replica.cc:330] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: stopping tablet replica
I20260808 18:18:14.172499 6404 tablet_service.cc:1514] Processing DeleteTablet for tablet e721bc2d9eb14ccea6c458da56909cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:14 UTC) from {username='oryx', principal='oryx/127.27.18.253@KRBTEST.COM'} at 127.0.0.1:49754
I20260808 18:18:14.172616 7106 raft_consensus.cc:2229] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:14.172655 7108 tablet_replica.cc:330] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: stopping tablet replica
I20260808 18:18:14.172761 7108 raft_consensus.cc:2229] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:14.172806 7106 raft_consensus.cc:2258] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:14.172887 7108 raft_consensus.cc:2258] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:14.173035 7109 tablet_replica.cc:330] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5: stopping tablet replica
I20260808 18:18:14.173156 7109 raft_consensus.cc:2229] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:14.173273 7109 raft_consensus.cc:2258] T e721bc2d9eb14ccea6c458da56909cc7 P c54b5b6fa5ed47bbaf7593e1991a10f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:14.173501 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 6180
I20260808 18:18:14.173671 7106 ts_tablet_manager.cc:1905] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:14.173688 7108 ts_tablet_manager.cc:1905] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:14.176385 7106 ts_tablet_manager.cc:1918] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:14.176530 7106 log.cc:1196] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/e721bc2d9eb14ccea6c458da56909cc7
I20260808 18:18:14.176633 7108 ts_tablet_manager.cc:1918] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:14.176712 7108 log.cc:1196] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/e721bc2d9eb14ccea6c458da56909cc7
I20260808 18:18:14.177021 7106 ts_tablet_manager.cc:1939] T e721bc2d9eb14ccea6c458da56909cc7 P 02e9a23d4880419185b4b55055c86ac4: Deleting consensus metadata
I20260808 18:18:14.177019 7108 ts_tablet_manager.cc:1939] T e721bc2d9eb14ccea6c458da56909cc7 P d26019add54840f7a3bd87db4626333c: Deleting consensus metadata
I20260808 18:18:14.177637 5941 catalog_manager.cc:4841] TS d26019add54840f7a3bd87db4626333c (127.27.18.194:33819): tablet e721bc2d9eb14ccea6c458da56909cc7 (table test-table [id=712edabaebb64c1e9c5d3805e12edc83]) successfully deleted
I20260808 18:18:14.178442 5941 catalog_manager.cc:4841] TS 02e9a23d4880419185b4b55055c86ac4 (127.27.18.195:42621): tablet e721bc2d9eb14ccea6c458da56909cc7 (table test-table [id=712edabaebb64c1e9c5d3805e12edc83]) successfully deleted
I20260808 18:18:14.185638 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 6520
I20260808 18:18:14.185827 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:14.185558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:37223 (local address 127.0.0.1:49758)
0808 18:18:14.185672 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:14.185745 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:37223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:18:14.187390 5941 catalog_manager.cc:4584] TS c54b5b6fa5ed47bbaf7593e1991a10f5 (127.27.18.193:37223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e721bc2d9eb14ccea6c458da56909cc7: Network error: Client connection negotiation failed: client connection to 127.27.18.193:37223: connect: Connection refused (error 111)
I20260808 18:18:14.201361 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 6716
I20260808 18:18:14.213578 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 5760
I20260808 18:18:14.222546 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 5911
2026-08-08T18:18:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5349 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:14 dist-test-slave-x4qh krb5kdc[7113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:14 dist-test-slave-x4qh krb5kdc[7113](info): set up 2 sockets
Aug 08 18:18:14 dist-test-slave-x4qh krb5kdc[7113](info): commencing operation
krb5kdc: starting...
W20260808 18:18:15.398880 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:15 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab.
Entry for principal oryx/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab.
I20260808 18:18:15.516304 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35849
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab
--principal=oryx/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:35849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:15.618676 7136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:15.618956 7136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:15.619024 7136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:15.622622 7136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:15.622692 7136 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 18:18:15.622723 7136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:15.622759 7136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:15.622795 7136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:15.622826 7136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:15.627561 7136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:35849
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.254.keytab
--principal=oryx/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:15.628440 7136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:15.629415 7136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:15.637346 7147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:15.637346 7143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:15.637564 7136 server_base.cc:1034] running on GCE node
W20260808 18:18:15.637380 7144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:15.638248 7136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:15.639216 7136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:15.640460 7136 hybrid_clock.cc:648] HybridClock initialized: now 1786213095640456 us; error 41 us; skew 500 ppm
Aug 08 18:18:15 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213095, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:15.643055 7136 init.cc:377] Logged in from keytab as oryx/127.27.18.254@KRBTEST.COM (short username oryx)
I20260808 18:18:15.643879 7136 webserver.cc:460] Webserver started at http://127.27.18.254:41073/ using document root <none> and password file <none>
I20260808 18:18:15.644384 7136 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:15.644471 7136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:15.644680 7136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:15.646498 7136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "9198006a60944565ae15fd1b4d505f9e"
format_stamp: "Formatted at 2026-08-08 18:18:15 on dist-test-slave-x4qh"
server_key: "46f17652d67d9c42f09106919d02bd81"
server_key_iv: "da9d63e8f00387cc04ba0e4b5383e524"
server_key_version: "encryptionkey@0"
I20260808 18:18:15.647074 7136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "9198006a60944565ae15fd1b4d505f9e"
format_stamp: "Formatted at 2026-08-08 18:18:15 on dist-test-slave-x4qh"
server_key: "46f17652d67d9c42f09106919d02bd81"
server_key_iv: "da9d63e8f00387cc04ba0e4b5383e524"
server_key_version: "encryptionkey@0"
I20260808 18:18:15.650641 7136 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:18:15.652884 7158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:15.654006 7136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:15.654132 7136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "9198006a60944565ae15fd1b4d505f9e"
format_stamp: "Formatted at 2026-08-08 18:18:15 on dist-test-slave-x4qh"
server_key: "46f17652d67d9c42f09106919d02bd81"
server_key_iv: "da9d63e8f00387cc04ba0e4b5383e524"
server_key_version: "encryptionkey@0"
I20260808 18:18:15.654229 7136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:15.669148 7136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:15.671717 7136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:15.671900 7136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:15.679337 7136 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:35849
I20260808 18:18:15.679342 7253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:35849 every 8 connection(s)
I20260808 18:18:15.680166 7136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:15.681751 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 7136
I20260808 18:18:15.681879 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:15.682178 27723 external_mini_cluster.cc:1428] Setting key 6cdb5c78fc57b668dabb2cbbb72897ab
I20260808 18:18:15.682960 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:15.689077 7254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Bootstrap starting.
Aug 08 18:18:15 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:15.691497 7254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:15.692425 7254 log.cc:824] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:15.694482 7254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: No bootstrap required, opened a new log
I20260808 18:18:15.696277 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:15.683431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.27.18.254:35849)
0808 18:18:15.683813 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:15.683828 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:15.683859 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:18:15.684429 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:15.684442 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:15.684713 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:15.684952 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:15.685816 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.686767 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.687898 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.688328 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.688444 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:15.691018 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:15.691023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:15.691034 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:15.691058 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:15.693454 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:15.693861 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:15.693866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:15.693871 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:15.694003 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:15.694245 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:15.694248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:15.694250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:15.694732 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:15.694917 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:15.695123 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 18:18:15.695439 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":128,"threads_started":1}
I20260808 18:18:15.698026 7254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9198006a60944565ae15fd1b4d505f9e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35849 } }
I20260808 18:18:15.698290 7254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:15.698361 7254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9198006a60944565ae15fd1b4d505f9e, State: Initialized, Role: FOLLOWER
I20260808 18:18:15.698784 7254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9198006a60944565ae15fd1b4d505f9e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35849 } }
I20260808 18:18:15.698915 7254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:15.698972 7254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:15.699061 7254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:15.699904 7254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9198006a60944565ae15fd1b4d505f9e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35849 } }
I20260808 18:18:15.700223 7254 leader_election.cc:302] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9198006a60944565ae15fd1b4d505f9e; no voters:
I20260808 18:18:15.700498 7254 leader_election.cc:288] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:15.700624 7265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:15.700874 7265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [term 1 LEADER]: Becoming Leader. State: Replica: 9198006a60944565ae15fd1b4d505f9e, State: Running, Role: LEADER
I20260808 18:18:15.701179 7265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9198006a60944565ae15fd1b4d505f9e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35849 } }
I20260808 18:18:15.701670 7254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:15.702243 7265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9198006a60944565ae15fd1b4d505f9e. Latest consensus state: current_term: 1 leader_uuid: "9198006a60944565ae15fd1b4d505f9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9198006a60944565ae15fd1b4d505f9e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35849 } } }
I20260808 18:18:15.702358 7265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:15.702765 7267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9198006a60944565ae15fd1b4d505f9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9198006a60944565ae15fd1b4d505f9e" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35849 } } }
I20260808 18:18:15.702919 7267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:15.702987 7274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:15.705605 7274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:15.710932 7274 catalog_manager.cc:1347] Generated new cluster ID: 452a75e8b4a7452f9b9ba3a800926821
I20260808 18:18:15.710990 7274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:15.724782 7274 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:15.725925 7274 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:15.734748 7274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Generated new TSK 0
I20260808 18:18:15.735407 7274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab.
Entry for principal oryx/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab.
I20260808 18:18:15.767735 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35849
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:15.871460 7295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:15.871717 7295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:15.871826 7295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:15.875833 7295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:15.875882 7295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:15.875941 7295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:18:15.876014 7295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:15.880730 7295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:35849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:15.881556 7295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:15.882462 7295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:18:15.891337 7295 server_base.cc:1034] running on GCE node
W20260808 18:18:15.891356 7307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:15.891319 7306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:15.891448 7313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:15.891894 7295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:15.892485 7295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:15.893657 7295 hybrid_clock.cc:648] HybridClock initialized: now 1786213095893630 us; error 38 us; skew 500 ppm
Aug 08 18:18:15 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213095, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:15.896267 7295 init.cc:377] Logged in from keytab as oryx/127.27.18.193@KRBTEST.COM (short username oryx)
I20260808 18:18:15.897039 7295 webserver.cc:460] Webserver started at http://127.27.18.193:45769/ using document root <none> and password file <none>
I20260808 18:18:15.897537 7295 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:15.897622 7295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:15.897835 7295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:15.899680 7295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "d99546e0bb7a432b9b2dab5485ab6edf"
format_stamp: "Formatted at 2026-08-08 18:18:15 on dist-test-slave-x4qh"
server_key: "f46fa6406578205801df80d0ae86c2a0"
server_key_iv: "ffd85c3d81ba6f5ead97c4ccd2cbc25c"
server_key_version: "encryptionkey@0"
I20260808 18:18:15.900200 7295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "d99546e0bb7a432b9b2dab5485ab6edf"
format_stamp: "Formatted at 2026-08-08 18:18:15 on dist-test-slave-x4qh"
server_key: "f46fa6406578205801df80d0ae86c2a0"
server_key_iv: "ffd85c3d81ba6f5ead97c4ccd2cbc25c"
server_key_version: "encryptionkey@0"
I20260808 18:18:15.903669 7295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:15.905797 7322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:15.906853 7295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:15.906980 7295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "d99546e0bb7a432b9b2dab5485ab6edf"
format_stamp: "Formatted at 2026-08-08 18:18:15 on dist-test-slave-x4qh"
server_key: "f46fa6406578205801df80d0ae86c2a0"
server_key_iv: "ffd85c3d81ba6f5ead97c4ccd2cbc25c"
server_key_version: "encryptionkey@0"
I20260808 18:18:15.907074 7295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:15.935842 7295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:15.938244 7295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:15.938412 7295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:15.939754 7295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:15.939816 7295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:15.939878 7295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:15.939924 7295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:18:15 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213095, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:15.950451 7295 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:33569
I20260808 18:18:15.950523 7502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:33569 every 8 connection(s)
I20260808 18:18:15.951545 7295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:15.952716 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:15.940391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:47471 (local address 127.27.18.254:35849)
0808 18:18:15.940532 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:15.940538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:15.941338 (+ 800us) server_negotiation.cc:407] Connection header received
0808 18:18:15.942406 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:15.942411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:15.942473 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:15.942532 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:15.944097 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.944944 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.946061 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.946314 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.946422 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:15.948829 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:15.948833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:15.948836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:15.948868 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:15.950553 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:15.951033 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:15.951036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:15.951038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:15.951129 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:15.951500 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:15.951506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:15.951511 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:15.951736 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:15.951848 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:15.952461 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 18:18:15.952580 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:18:15.953307 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 7295
I20260808 18:18:15.953317 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:15.940717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.27.18.193:47471)
0808 18:18:15.941179 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:15.941231 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:15.942165 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:15.943000 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:15.943010 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:15.943488 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:15.943969 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:15.943983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.945079 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.945082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:15.945928 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:15.945938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.946472 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.946477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:15.946642 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:15.947230 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:15.947248 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:15.948680 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:15.950667 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:15.950672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:15.950687 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:15.950933 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:15.951230 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:15.951232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:15.951233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:15.951356 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:15.951849 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:15.951854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:15.952132 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 18:18:15.952339 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 18:18:15.952594 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":128,"threads_started":1}
I20260808 18:18:15.953445 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:15.953717 27723 external_mini_cluster.cc:1428] Setting key de458c6a4f520a722bf5aafa84ace88a
I20260808 18:18:15.954557 7504 heartbeater.cc:347] Connected to a master server at 127.27.18.254:35849
I20260808 18:18:15.954681 7504 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:15.955181 7504 heartbeater.cc:511] Master 127.27.18.254:35849 requested a full tablet report, sending...
I20260808 18:18:15.956818 7187 ts_manager.cc:194] Registered new tserver with Master: d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569)
I20260808 18:18:15.958041 7187 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:47471
I20260808 18:18:15.964224 7514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:15.956661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.27.18.193:33613)
0808 18:18:15.957026 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:15.957039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:15.957129 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:15.957452 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:15.957456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:15.957652 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:15.957773 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:15.957782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.958646 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.958650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:15.959509 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:15.959522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.960034 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.960039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:15.960163 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:15.960600 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:15.960618 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:15.960997 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:15.962908 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:15.962910 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:15.962912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:15.963079 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:15.963339 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:15.963341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:15.963342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:15.963454 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:15.963868 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:15.963870 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:15.963935 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 18:18:15.964007 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:18:15.964094 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":210,"threads_started":1}
I20260808 18:18:15.964357 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:15.956719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:33613 (local address 127.27.18.254:35849)
0808 18:18:15.956811 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:15.956814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:15.957096 (+ 282us) server_negotiation.cc:407] Connection header received
0808 18:18:15.957214 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:15.957217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:15.957253 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:15.957339 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:15.957905 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.958520 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.959639 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:15.959916 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:15.960011 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:15.961161 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:15.961167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:15.961174 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:15.961204 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:15.962795 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:15.963170 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:15.963175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:15.963176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:15.963257 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:15.963544 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:15.963549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:15.963552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:15.963768 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:15.963881 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:15.964088 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 18:18:15.964200 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":29}
WARNING: no policy specified for oryx/127.27.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab.
Entry for principal oryx/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab.
I20260808 18:18:15.980239 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35849
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:16.087297 7518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:16.087636 7518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:16.087731 7518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:16.091892 7518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:16.091944 7518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:16.091984 7518 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:18:16.092025 7518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:16.097411 7518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:35849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:16.098296 7518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:16.099156 7518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:16.108498 7531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:16.108546 7527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:16.108561 7528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:16.108777 7518 server_base.cc:1034] running on GCE node
I20260808 18:18:16.109175 7518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:16.109735 7518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:16.110910 7518 hybrid_clock.cc:648] HybridClock initialized: now 1786213096110883 us; error 39 us; skew 500 ppm
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:16.113440 7518 init.cc:377] Logged in from keytab as oryx/127.27.18.194@KRBTEST.COM (short username oryx)
I20260808 18:18:16.114229 7518 webserver.cc:460] Webserver started at http://127.27.18.194:42479/ using document root <none> and password file <none>
I20260808 18:18:16.114730 7518 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:16.114813 7518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:16.115032 7518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:16.117027 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "08025cae082f46c2b28f078311a09f97"
format_stamp: "Formatted at 2026-08-08 18:18:16 on dist-test-slave-x4qh"
server_key: "9d327e56195198161fa6fe5019874f83"
server_key_iv: "83f38bf6abee7bbc98596730435334f5"
server_key_version: "encryptionkey@0"
I20260808 18:18:16.117538 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "08025cae082f46c2b28f078311a09f97"
format_stamp: "Formatted at 2026-08-08 18:18:16 on dist-test-slave-x4qh"
server_key: "9d327e56195198161fa6fe5019874f83"
server_key_iv: "83f38bf6abee7bbc98596730435334f5"
server_key_version: "encryptionkey@0"
I20260808 18:18:16.120788 7518 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 18:18:16.122979 7542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:16.124049 7518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:18:16.124166 7518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "08025cae082f46c2b28f078311a09f97"
format_stamp: "Formatted at 2026-08-08 18:18:16 on dist-test-slave-x4qh"
server_key: "9d327e56195198161fa6fe5019874f83"
server_key_iv: "83f38bf6abee7bbc98596730435334f5"
server_key_version: "encryptionkey@0"
I20260808 18:18:16.124258 7518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:16.153123 7518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:16.156122 7518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:16.156307 7518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:16.157684 7518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:16.157753 7518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:16.157819 7518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:16.157863 7518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:16.168226 7518 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:41633
I20260808 18:18:16.168290 7718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:41633 every 8 connection(s)
I20260808 18:18:16.169152 7518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:16.170588 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.158441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:56697 (local address 127.27.18.254:35849)
0808 18:18:16.158578 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.158581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.159524 (+ 943us) server_negotiation.cc:407] Connection header received
0808 18:18:16.160438 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.160441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.160499 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.160573 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.162111 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.162729 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.164403 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.164737 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.164859 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.167061 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.167064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.167066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.167087 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.168610 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.169049 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.169052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.169054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.169144 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.169513 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.169516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.169518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.169742 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.169836 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.170390 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.170481 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:18:16.171268 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.158911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.27.18.194:56697)
0808 18:18:16.159355 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.159435 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.160240 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.160757 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.160767 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.161281 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:16.161931 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.161956 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.162868 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.162874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.164255 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.164266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.164900 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.164904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.165051 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.165551 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:16.165566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:16.166906 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:16.168711 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.168716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.168724 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.168950 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.169243 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.169246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.169248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.169402 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.169892 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:16.169898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:16.170146 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.170335 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.170544 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":142,"threads_started":1}
I20260808 18:18:16.172268 7719 heartbeater.cc:347] Connected to a master server at 127.27.18.254:35849
I20260808 18:18:16.172402 7719 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:16.172879 7719 heartbeater.cc:511] Master 127.27.18.254:35849 requested a full tablet report, sending...
I20260808 18:18:16.173889 7188 ts_manager.cc:194] Registered new tserver with Master: 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633)
I20260808 18:18:16.174598 7188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:56697
I20260808 18:18:16.176127 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 7518
I20260808 18:18:16.176275 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:16.176582 27723 external_mini_cluster.cc:1428] Setting key b718547c337bb23c358cd47a33ad65a9
I20260808 18:18:16.181317 7732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.174283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.27.18.194:47059)
0808 18:18:16.174547 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.174557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.174625 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.174850 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.174853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.175038 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:16.175145 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.175151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.176138 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.176143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.176941 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.176956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.177477 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.177480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.177573 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.178047 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:16.178061 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:16.178450 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:16.179992 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.179998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.180001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.180207 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.180475 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.180477 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.180478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.180543 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.180980 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:16.180982 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:16.181045 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.181118 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.181204 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260808 18:18:16.181504 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.174337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:47059 (local address 127.27.18.254:35849)
0808 18:18:16.174444 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.174448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.174615 (+ 167us) server_negotiation.cc:407] Connection header received
0808 18:18:16.174703 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.174706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.174739 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.174814 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.175250 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.175983 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.177061 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.177313 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.177479 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.178581 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.178583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.178585 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.178605 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.179860 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.180295 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.180299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.180302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.180388 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.180627 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.180631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.180634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.180896 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.180993 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.181204 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.181321 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab.
Entry for principal oryx/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab.
I20260808 18:18:16.202071 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35849
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:16.306038 7736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:16.306244 7736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:16.306316 7736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:16.310055 7736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:16.310110 7736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:16.310169 7736 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 18:18:16.310215 7736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:16.314800 7736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.18.254:35849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:16.315682 7736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:16.316558 7736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:18:16.325487 7736 server_base.cc:1034] running on GCE node
W20260808 18:18:16.325448 7752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:16.325443 7750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:16.325503 7748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:16.326146 7736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:16.326768 7736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:16.327919 7736 hybrid_clock.cc:648] HybridClock initialized: now 1786213096327906 us; error 24 us; skew 500 ppm
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:16.330479 7736 init.cc:377] Logged in from keytab as oryx/127.27.18.195@KRBTEST.COM (short username oryx)
I20260808 18:18:16.331169 7736 webserver.cc:460] Webserver started at http://127.27.18.195:34867/ using document root <none> and password file <none>
I20260808 18:18:16.331707 7736 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:16.331766 7736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:16.331915 7736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:16.333652 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "4c2735609afa4a7db97766090f039949"
format_stamp: "Formatted at 2026-08-08 18:18:16 on dist-test-slave-x4qh"
server_key: "053fa24c3d0cc8e0aaaad13f99e7661f"
server_key_iv: "9601fe80c1cd3357cade33cb59488f6b"
server_key_version: "encryptionkey@0"
I20260808 18:18:16.334118 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "4c2735609afa4a7db97766090f039949"
format_stamp: "Formatted at 2026-08-08 18:18:16 on dist-test-slave-x4qh"
server_key: "053fa24c3d0cc8e0aaaad13f99e7661f"
server_key_iv: "9601fe80c1cd3357cade33cb59488f6b"
server_key_version: "encryptionkey@0"
I20260808 18:18:16.337352 7736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:18:16.339347 7764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:16.340453 7736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:16.340550 7736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "4c2735609afa4a7db97766090f039949"
format_stamp: "Formatted at 2026-08-08 18:18:16 on dist-test-slave-x4qh"
server_key: "053fa24c3d0cc8e0aaaad13f99e7661f"
server_key_iv: "9601fe80c1cd3357cade33cb59488f6b"
server_key_version: "encryptionkey@0"
I20260808 18:18:16.340626 7736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:16.352200 7736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:16.354488 7736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:16.354622 7736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:16.356040 7736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:16.356087 7736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:16.356124 7736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:16.356173 7736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:16.366269 7736 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:35313
I20260808 18:18:16.366348 7946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:35313 every 8 connection(s)
I20260808 18:18:16.367233 7736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:16.369882 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.356630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:52741 (local address 127.27.18.254:35849)
0808 18:18:16.356804 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.356811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.358003 (+ 1192us) server_negotiation.cc:407] Connection header received
0808 18:18:16.359241 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.359245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.359303 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.359414 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.360823 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.361746 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.363483 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.363708 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.363815 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.366457 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.366461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.366464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.366490 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.368015 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.368446 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.368450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.368452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.368533 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.368873 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.368876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.368878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.369046 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.369155 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.369597 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.369710 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:18:16.370558 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.357386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.27.18.195:52741)
0808 18:18:16.357870 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.357919 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.359011 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.359575 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.359582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.360040 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:16.360662 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.360684 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.361911 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.361916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.363337 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.363353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.363859 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.363863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.364022 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.364556 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:16.364580 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:16.366297 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:16.368114 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.368118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.368127 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.368345 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.368638 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.368641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.368643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.368774 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.369169 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:16.369175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:16.369387 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.369553 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.369831 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":130,"threads_started":1}
I20260808 18:18:16.371608 7950 heartbeater.cc:347] Connected to a master server at 127.27.18.254:35849
I20260808 18:18:16.371739 7950 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:16.372195 7950 heartbeater.cc:511] Master 127.27.18.254:35849 requested a full tablet report, sending...
I20260808 18:18:16.373236 7187 ts_manager.cc:194] Registered new tserver with Master: 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:16.373983 7187 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:52741
I20260808 18:18:16.377486 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 7736
I20260808 18:18:16.377628 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:16.377970 27723 external_mini_cluster.cc:1428] Setting key 2f1588661726e2ca8080fb15b3cd4c35
I20260808 18:18:16.379966 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:18:16.380066 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.373753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:36471 (local address 127.27.18.254:35849)
0808 18:18:16.373848 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.373851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.373864 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:16.373946 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.373948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.373977 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.374035 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.374469 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.374963 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.375975 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.376165 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.376235 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.377114 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.377117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.377119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.377139 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.378502 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.378911 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.378913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.378914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.378982 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.379251 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.379254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.379256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.379484 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.379582 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.379842 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.379943 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 18:18:16.380076 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.373525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.27.18.195:36471)
0808 18:18:16.373774 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.373786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.373860 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.374072 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.374075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.374273 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:16.374367 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.374372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.375100 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.375103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.375871 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.375880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.376248 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.376251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.376314 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.376715 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:16.376726 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:16.377006 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:16.378609 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.378611 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.378630 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.378838 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.379087 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.379089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.379091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.379160 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.379574 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:16.379577 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:16.379669 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.379769 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.379903 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":90,"threads_started":1}
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:18:16.402863 7261 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:16.397251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.27.18.254:35849)
0808 18:18:16.397387 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.397391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.397406 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:18:16.397435 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.397436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.397466 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.397540 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.398168 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.398635 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.399486 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.399675 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.399770 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.402741 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59464: BlockingRecv error: recv got EOF from 127.0.0.1:59464 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:16.411537 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.404223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.27.18.254:35849)
0808 18:18:16.404347 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.404352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.404493 (+ 141us) server_negotiation.cc:407] Connection header received
0808 18:18:16.404610 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.404613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.404649 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.404731 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.405277 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.405797 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.406529 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.406726 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.406828 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.409002 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.409005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.409007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.409028 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.410188 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.410521 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.410523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.410524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.410586 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.410806 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.410808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.410809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.410949 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.411052 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.411260 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.411344 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:18:16.413985 7187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:16.416885 7187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:16.429983 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.424840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.27.18.194:41633)
0808 18:18:16.425388 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.425399 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.425417 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:18:16.425477 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.425481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.425617 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.425773 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.426150 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.427032 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.428800 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.429322 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.429441 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.429602 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.429755 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.429848 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":138,"threads_started":1}
I20260808 18:18:16.430150 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.424009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35313 (local address 127.0.0.1:55334)
0808 18:18:16.424477 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.424520 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.424638 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.425769 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.425774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.425794 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.425893 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.425897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.427262 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.427265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.428640 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.428668 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.429703 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.429706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.429836 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.429863 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.429920 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.429978 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":139,"thread_start_us":66,"threads_started":1}
I20260808 18:18:16.429996 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.424650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:41633 (local address 127.0.0.1:33818)
0808 18:18:16.424968 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.424980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.425069 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.425985 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.425986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.425997 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.426050 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.426053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.427124 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.427128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.428680 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.428686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.429452 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.429456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.429568 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.429609 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.429725 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.429880 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":157,"thread_start_us":88,"threads_started":1}
I20260808 18:18:16.430449 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.424980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55412 (local address 127.27.18.193:33569)
0808 18:18:16.426044 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.426058 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.426082 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:18:16.426150 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.426155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.426334 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.426475 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.426684 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.427583 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.428839 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.429572 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.429680 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.429822 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.430210 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.430324 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":81,"threads_started":1}
I20260808 18:18:16.430464 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.424106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.27.18.195:35313)
0808 18:18:16.425311 (+ 1205us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.425325 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.425348 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:18:16.425422 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.425427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.425615 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.425773 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.426004 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.427171 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.428822 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.429572 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.430082 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.430235 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.430273 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.430354 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":92,"threads_started":1}
I20260808 18:18:16.431499 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.424106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33569 (local address 127.0.0.1:55412)
0808 18:18:16.424905 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.424919 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.425012 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.426472 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.426475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.426494 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.426567 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.426572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.427702 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.427705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.428712 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.428722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.429877 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.429881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.429965 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.429979 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.430052 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.430104 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":57,"thread_start_us":83,"threads_started":1}
I20260808 18:18:16.433117 7623 tablet_service.cc:1467] Processing CreateTablet for tablet efa87fc81c9d440ca61ac3368253fadc (DEFAULT_TABLE table=test-table [id=a73a0b26ccef4cc5929d5376709a5302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.433115 7842 tablet_service.cc:1467] Processing CreateTablet for tablet efa87fc81c9d440ca61ac3368253fadc (DEFAULT_TABLE table=test-table [id=a73a0b26ccef4cc5929d5376709a5302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.433622 7400 tablet_service.cc:1467] Processing CreateTablet for tablet efa87fc81c9d440ca61ac3368253fadc (DEFAULT_TABLE table=test-table [id=a73a0b26ccef4cc5929d5376709a5302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.434134 7623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa87fc81c9d440ca61ac3368253fadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.434137 7842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa87fc81c9d440ca61ac3368253fadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.434419 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa87fc81c9d440ca61ac3368253fadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.440254 7993 tablet_bootstrap.cc:492] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Bootstrap starting.
I20260808 18:18:16.441888 7997 tablet_bootstrap.cc:492] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Bootstrap starting.
I20260808 18:18:16.442385 7993 tablet_bootstrap.cc:654] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.443051 7993 log.cc:824] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:16.443594 7997 tablet_bootstrap.cc:654] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.443780 7994 tablet_bootstrap.cc:492] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Bootstrap starting.
I20260808 18:18:16.444485 7997 log.cc:824] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:16.445902 7994 tablet_bootstrap.cc:654] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.446143 7993 tablet_bootstrap.cc:492] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: No bootstrap required, opened a new log
I20260808 18:18:16.446393 7993 ts_tablet_manager.cc:1397] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:18:16.446827 7994 log.cc:824] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:16.447238 7997 tablet_bootstrap.cc:492] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: No bootstrap required, opened a new log
I20260808 18:18:16.447472 7997 ts_tablet_manager.cc:1397] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:18:16.450726 7993 raft_consensus.cc:348] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.450999 7993 raft_consensus.cc:374] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.451061 7993 raft_consensus.cc:729] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.451507 7993 consensus_queue.cc:260] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.451630 7997 raft_consensus.cc:348] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.451879 7997 raft_consensus.cc:374] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.451928 7997 raft_consensus.cc:729] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08025cae082f46c2b28f078311a09f97, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.452486 7993 ts_tablet_manager.cc:1428] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:18:16.452409 7997 consensus_queue.cc:260] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.452834 7504 heartbeater.cc:503] Master 127.27.18.254:35849 was elected leader, sending a full tablet report...
I20260808 18:18:16.454648 7719 heartbeater.cc:503] Master 127.27.18.254:35849 was elected leader, sending a full tablet report...
I20260808 18:18:16.454962 7997 ts_tablet_manager.cc:1428] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 18:18:16.457067 7994 tablet_bootstrap.cc:492] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: No bootstrap required, opened a new log
I20260808 18:18:16.457283 7994 ts_tablet_manager.cc:1397] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
W20260808 18:18:16.457789 7505 tablet.cc:2369] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:16.461824 7994 raft_consensus.cc:348] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.462076 7994 raft_consensus.cc:374] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.462141 7994 raft_consensus.cc:729] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2735609afa4a7db97766090f039949, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.462571 7994 consensus_queue.cc:260] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.463291 7950 heartbeater.cc:503] Master 127.27.18.254:35849 was elected leader, sending a full tablet report...
I20260808 18:18:16.463594 7994 ts_tablet_manager.cc:1428] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:18:16.510535 8009 raft_consensus.cc:482] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:16.510737 8009 raft_consensus.cc:504] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.511864 8009 leader_election.cc:288] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313), d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569)
I20260808 18:18:16.515763 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.512147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:59041 (local address 127.27.18.195:35313)
0808 18:18:16.512255 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.512260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.512635 (+ 375us) server_negotiation.cc:407] Connection header received
0808 18:18:16.512772 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.512775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.512828 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.512901 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.513182 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.513743 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.514695 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.515258 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.515351 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.515446 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.515614 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.515669 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:18:16.516008 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.512083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35313 (local address 127.27.18.194:59041)
0808 18:18:16.512537 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.512553 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.512670 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.512934 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.512937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.512955 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.513075 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.513081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.513852 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.513854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.514577 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.514585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.515361 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.515364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.515495 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.515522 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.515599 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.515664 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":107,"spinlock_wait_cycles":768,"thread_start_us":107,"threads_started":1}
I20260808 18:18:16.516271 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.512083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33569 (local address 127.27.18.194:42415)
0808 18:18:16.512194 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.512210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.512299 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.512576 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.512578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.512596 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.512759 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.512766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.513592 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.513595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.515047 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.515057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.515851 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.515855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.515969 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.515990 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.516059 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.516160 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"spinlock_wait_cycles":8960}
I20260808 18:18:16.516310 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.512146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:42415 (local address 127.27.18.193:33569)
0808 18:18:16.512255 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.512260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.512276 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:18:16.512421 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.512424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.512478 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.512567 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.512892 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.513492 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.515172 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.515730 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.515837 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.515899 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.516106 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.516193 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:18:16.516687 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa87fc81c9d440ca61ac3368253fadc" candidate_uuid: "08025cae082f46c2b28f078311a09f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949" is_pre_election: true
I20260808 18:18:16.516770 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa87fc81c9d440ca61ac3368253fadc" candidate_uuid: "08025cae082f46c2b28f078311a09f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" is_pre_election: true
I20260808 18:18:16.516963 7876 raft_consensus.cc:2454] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08025cae082f46c2b28f078311a09f97 in term 0.
I20260808 18:18:16.517035 7434 raft_consensus.cc:2454] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08025cae082f46c2b28f078311a09f97 in term 0.
I20260808 18:18:16.517388 7548 leader_election.cc:302] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, 4c2735609afa4a7db97766090f039949; no voters:
I20260808 18:18:16.517647 8009 raft_consensus.cc:2781] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:16.517714 8009 raft_consensus.cc:482] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:16.517757 8009 raft_consensus.cc:3037] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.518778 8009 raft_consensus.cc:504] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.519088 8009 leader_election.cc:288] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313), d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569)
I20260808 18:18:16.519428 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa87fc81c9d440ca61ac3368253fadc" candidate_uuid: "08025cae082f46c2b28f078311a09f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99546e0bb7a432b9b2dab5485ab6edf"
I20260808 18:18:16.519440 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa87fc81c9d440ca61ac3368253fadc" candidate_uuid: "08025cae082f46c2b28f078311a09f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949"
I20260808 18:18:16.519515 7434 raft_consensus.cc:3037] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.519544 7876 raft_consensus.cc:3037] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.520412 7434 raft_consensus.cc:2454] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08025cae082f46c2b28f078311a09f97 in term 1.
I20260808 18:18:16.520500 7876 raft_consensus.cc:2454] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08025cae082f46c2b28f078311a09f97 in term 1.
I20260808 18:18:16.520690 7546 leader_election.cc:302] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:16.520856 8009 raft_consensus.cc:2781] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:16.521018 8009 raft_consensus.cc:686] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 1 LEADER]: Becoming Leader. State: Replica: 08025cae082f46c2b28f078311a09f97, State: Running, Role: LEADER
I20260808 18:18:16.521520 8009 consensus_queue.cc:237] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } }
I20260808 18:18:16.524648 7187 catalog_manager.cc:5495] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08025cae082f46c2b28f078311a09f97 (127.27.18.194). New cstate: current_term: 1 leader_uuid: "08025cae082f46c2b28f078311a09f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } health_report { overall_health: HEALTHY } } }
I20260808 18:18:16.553311 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.550385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.27.18.194:41633)
0808 18:18:16.550485 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.550488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.550499 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:18:16.550617 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.550620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.550665 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.550745 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:16.550934 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.551402 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.552128 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.552326 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.552394 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.552634 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.553003 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.553085 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.553163 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.553206 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:18:16.558632 7434 raft_consensus.cc:1261] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Refusing update from remote peer 08025cae082f46c2b28f078311a09f97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:16.558983 7876 raft_consensus.cc:1261] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Refusing update from remote peer 08025cae082f46c2b28f078311a09f97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:16.559248 8009 consensus_queue.cc:1035] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:16.559515 8014 consensus_queue.cc:1035] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:16.570115 8023 mvcc.cc:205] Tried to move back new op lower bound from 7316328843497693184 to 7316328843354656768. Current Snapshot: MvccSnapshot[applied={T|T < 7316328843497693184}]
I20260808 18:18:16.573120 7188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:16.573282 7188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:16.575518 7188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Sending DeleteTablet for 3 replicas of tablet efa87fc81c9d440ca61ac3368253fadc
I20260808 18:18:16.576182 7842 tablet_service.cc:1514] Processing DeleteTablet for tablet efa87fc81c9d440ca61ac3368253fadc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:16 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55334
I20260808 18:18:16.576263 7623 tablet_service.cc:1514] Processing DeleteTablet for tablet efa87fc81c9d440ca61ac3368253fadc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:16 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33818
I20260808 18:18:16.576653 8035 tablet_replica.cc:330] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: stopping tablet replica
I20260808 18:18:16.576642 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet efa87fc81c9d440ca61ac3368253fadc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:16 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55412
I20260808 18:18:16.576960 8035 raft_consensus.cc:2229] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:16.577109 8037 tablet_replica.cc:330] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: stopping tablet replica
I20260808 18:18:16.577241 8035 raft_consensus.cc:2258] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:16.577445 8037 raft_consensus.cc:2229] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:16.577710 7188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:16.578069 7188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:16.578213 8037 raft_consensus.cc:2258] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:16.578519 8035 ts_tablet_manager.cc:1905] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:16.579597 8037 ts_tablet_manager.cc:1905] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:16.581065 8035 ts_tablet_manager.cc:1918] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:16.581164 8035 log.cc:1196] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/efa87fc81c9d440ca61ac3368253fadc
I20260808 18:18:16.581481 8035 ts_tablet_manager.cc:1939] T efa87fc81c9d440ca61ac3368253fadc P 4c2735609afa4a7db97766090f039949: Deleting consensus metadata
I20260808 18:18:16.582300 8037 ts_tablet_manager.cc:1918] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:16.582451 8037 log.cc:1196] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/efa87fc81c9d440ca61ac3368253fadc
I20260808 18:18:16.582810 8037 ts_tablet_manager.cc:1939] T efa87fc81c9d440ca61ac3368253fadc P d99546e0bb7a432b9b2dab5485ab6edf: Deleting consensus metadata
I20260808 18:18:16.583079 7164 catalog_manager.cc:4841] TS 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313): tablet efa87fc81c9d440ca61ac3368253fadc (table test-table [id=a73a0b26ccef4cc5929d5376709a5302]) successfully deleted
I20260808 18:18:16.583272 7623 tablet_service.cc:1467] Processing CreateTablet for tablet b6656f51020b40ec8545b8a04b5447c5 (DEFAULT_TABLE table=test-table [id=2a086061b64245989135a7bc65b23702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.583535 7842 tablet_service.cc:1467] Processing CreateTablet for tablet b6656f51020b40ec8545b8a04b5447c5 (DEFAULT_TABLE table=test-table [id=2a086061b64245989135a7bc65b23702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.583642 7623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6656f51020b40ec8545b8a04b5447c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.583763 7842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6656f51020b40ec8545b8a04b5447c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.583844 7160 catalog_manager.cc:4841] TS d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569): tablet efa87fc81c9d440ca61ac3368253fadc (table test-table [id=a73a0b26ccef4cc5929d5376709a5302]) successfully deleted
I20260808 18:18:16.585520 7997 tablet_bootstrap.cc:492] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Bootstrap starting.
I20260808 18:18:16.585904 7994 tablet_bootstrap.cc:492] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Bootstrap starting.
I20260808 18:18:16.585918 7400 tablet_service.cc:1467] Processing CreateTablet for tablet b6656f51020b40ec8545b8a04b5447c5 (DEFAULT_TABLE table=test-table [id=2a086061b64245989135a7bc65b23702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.586162 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6656f51020b40ec8545b8a04b5447c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.586571 7997 tablet_bootstrap.cc:654] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.586830 7994 tablet_bootstrap.cc:654] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.587950 7994 tablet_bootstrap.cc:492] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: No bootstrap required, opened a new log
I20260808 18:18:16.588025 7994 ts_tablet_manager.cc:1397] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:16.588421 7993 tablet_bootstrap.cc:492] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Bootstrap starting.
I20260808 18:18:16.588455 7994 raft_consensus.cc:348] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.588555 7997 tablet_bootstrap.cc:492] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: No bootstrap required, opened a new log
I20260808 18:18:16.588567 7994 raft_consensus.cc:374] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.588605 7994 raft_consensus.cc:729] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2735609afa4a7db97766090f039949, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.588630 7997 ts_tablet_manager.cc:1397] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:18:16.588739 7994 consensus_queue.cc:260] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.588997 7994 ts_tablet_manager.cc:1428] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.589062 7997 raft_consensus.cc:348] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.589195 7997 raft_consensus.cc:374] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.589246 7997 raft_consensus.cc:729] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08025cae082f46c2b28f078311a09f97, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.589350 7993 tablet_bootstrap.cc:654] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.589398 7997 consensus_queue.cc:260] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.589704 7997 ts_tablet_manager.cc:1428] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.591403 8036 tablet_replica.cc:330] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: stopping tablet replica
I20260808 18:18:16.591672 8036 raft_consensus.cc:2229] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:16.591992 8036 raft_consensus.cc:2258] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:16.593430 8036 ts_tablet_manager.cc:1905] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:16.595834 8036 ts_tablet_manager.cc:1918] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:16.595923 8036 log.cc:1196] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/efa87fc81c9d440ca61ac3368253fadc
I20260808 18:18:16.596241 8036 ts_tablet_manager.cc:1939] T efa87fc81c9d440ca61ac3368253fadc P 08025cae082f46c2b28f078311a09f97: Deleting consensus metadata
I20260808 18:18:16.597184 7160 catalog_manager.cc:4841] TS 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633): tablet efa87fc81c9d440ca61ac3368253fadc (table test-table [id=a73a0b26ccef4cc5929d5376709a5302]) successfully deleted
I20260808 18:18:16.600031 7993 tablet_bootstrap.cc:492] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: No bootstrap required, opened a new log
I20260808 18:18:16.600123 7993 ts_tablet_manager.cc:1397] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260808 18:18:16.600651 7993 raft_consensus.cc:348] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.600751 7993 raft_consensus.cc:374] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.600791 7993 raft_consensus.cc:729] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.600916 7993 consensus_queue.cc:260] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.601142 7993 ts_tablet_manager.cc:1428] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.603480 8008 raft_consensus.cc:482] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:16.603574 8008 raft_consensus.cc:504] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.604657 8008 leader_election.cc:288] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633), 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:16.608636 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.604959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:47571 (local address 127.27.18.195:35313)
0808 18:18:16.605084 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.605089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.605489 (+ 400us) server_negotiation.cc:407] Connection header received
0808 18:18:16.605593 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.605596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.605631 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.605712 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.606002 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.606477 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.607676 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.608100 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.608166 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.608203 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.608487 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.608550 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:18:16.608659 8044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.604898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35313 (local address 127.27.18.193:47571)
0808 18:18:16.605396 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.605415 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.605518 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.605723 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.605726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.605753 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.605899 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.605904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.606581 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.606586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.607540 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.607550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.608287 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.608289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.608367 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.608383 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.608455 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.608519 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":27,"thread_start_us":107,"threads_started":1}
I20260808 18:18:16.609084 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6656f51020b40ec8545b8a04b5447c5" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949" is_pre_election: true
I20260808 18:18:16.609216 7876 raft_consensus.cc:2454] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 0.
I20260808 18:18:16.609241 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.605158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:44111 (local address 127.27.18.194:41633)
0808 18:18:16.605260 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.605264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.605276 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:16.605388 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.605391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.605447 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.605522 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.605934 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.606678 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.608110 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.608653 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.608730 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.608771 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.609078 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.609142 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:18:16.609608 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.604798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:41633 (local address 127.27.18.193:44111)
0808 18:18:16.605151 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.605164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.605260 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.605628 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.605631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.605649 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.605798 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.605805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.606813 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.606816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.607979 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.607990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.608858 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.608861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.608956 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.608975 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.609385 (+ 410us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.609449 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260808 18:18:16.609629 7327 leader_election.cc:302] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2735609afa4a7db97766090f039949, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:16.609957 8008 raft_consensus.cc:2781] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:16.610044 8008 raft_consensus.cc:482] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:16.610095 8008 raft_consensus.cc:3037] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.610085 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6656f51020b40ec8545b8a04b5447c5" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97" is_pre_election: true
I20260808 18:18:16.610270 7656 raft_consensus.cc:2454] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 0.
I20260808 18:18:16.610993 8008 raft_consensus.cc:504] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.611297 8008 leader_election.cc:288] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 election: Requested vote from peers 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633), 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:16.611594 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6656f51020b40ec8545b8a04b5447c5" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949"
I20260808 18:18:16.611644 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6656f51020b40ec8545b8a04b5447c5" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97"
I20260808 18:18:16.611699 7876 raft_consensus.cc:3037] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.611747 7656 raft_consensus.cc:3037] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.612437 7876 raft_consensus.cc:2454] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 1.
I20260808 18:18:16.612713 7327 leader_election.cc:302] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2735609afa4a7db97766090f039949, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:16.612736 7656 raft_consensus.cc:2454] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 1.
I20260808 18:18:16.613109 8008 raft_consensus.cc:2781] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:16.613344 8008 raft_consensus.cc:686] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 1 LEADER]: Becoming Leader. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Running, Role: LEADER
I20260808 18:18:16.613574 8008 consensus_queue.cc:237] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.615341 7188 catalog_manager.cc:5495] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf reported cstate change: term changed from 0 to 1, leader changed from <none> to d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193). New cstate: current_term: 1 leader_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } health_report { overall_health: UNKNOWN } } }
W20260808 18:18:16.618090 7951 tablet.cc:2369] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213095, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.254@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:16.653008 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.645086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35849 (local address 127.0.0.1:59490)
0808 18:18:16.645356 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.645368 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.645494 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.645716 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.645718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.646106 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:16.646192 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.646198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.646919 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.646924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.647519 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.647529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.647948 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.647951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.648031 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.648612 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:16.648624 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:16.649966 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:16.651484 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.651493 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.651499 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.651727 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.652006 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:16.652010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:16.652011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:16.652089 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:16.652443 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:16.652450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:16.652557 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.652666 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.652821 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
I20260808 18:18:16.653008 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.645159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.27.18.254:35849)
0808 18:18:16.645282 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.645286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.645452 (+ 166us) server_negotiation.cc:407] Connection header received
0808 18:18:16.645577 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.645579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.645617 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.645698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.646316 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.646805 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.647628 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.647845 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.647945 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.650090 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.650093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.650095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.650116 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.651332 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.651840 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.651843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.651845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.651922 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.652171 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.652176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.652178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.652344 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.652444 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.652710 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.652888 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":43}
I20260808 18:18:16.657099 7188 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:59490:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 18:18:16.657878 7188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:16.662369 7400 tablet_service.cc:1467] Processing CreateTablet for tablet 75d865dacafb44649c9aa7d73febf2db (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32db5e0b4dea4bf8b348a7c405dc4487]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:18:16.662364 7623 tablet_service.cc:1467] Processing CreateTablet for tablet 75d865dacafb44649c9aa7d73febf2db (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32db5e0b4dea4bf8b348a7c405dc4487]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:18:16.662504 7842 tablet_service.cc:1467] Processing CreateTablet for tablet 75d865dacafb44649c9aa7d73febf2db (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32db5e0b4dea4bf8b348a7c405dc4487]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 18:18:16.662762 7623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d865dacafb44649c9aa7d73febf2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.662762 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d865dacafb44649c9aa7d73febf2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.662878 7842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d865dacafb44649c9aa7d73febf2db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.665522 7997 tablet_bootstrap.cc:492] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: Bootstrap starting.
I20260808 18:18:16.665665 7993 tablet_bootstrap.cc:492] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: Bootstrap starting.
I20260808 18:18:16.666427 7997 tablet_bootstrap.cc:654] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.666446 7993 tablet_bootstrap.cc:654] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.667527 7994 tablet_bootstrap.cc:492] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: Bootstrap starting.
I20260808 18:18:16.668282 7997 tablet_bootstrap.cc:492] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: No bootstrap required, opened a new log
I20260808 18:18:16.668365 7997 ts_tablet_manager.cc:1397] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:18:16.668459 7994 tablet_bootstrap.cc:654] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.668756 7997 raft_consensus.cc:348] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.668900 7997 raft_consensus.cc:374] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.668927 7997 raft_consensus.cc:729] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08025cae082f46c2b28f078311a09f97, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.669031 7997 consensus_queue.cc:260] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.669247 7997 ts_tablet_manager.cc:1428] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.669574 7993 tablet_bootstrap.cc:492] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: No bootstrap required, opened a new log
I20260808 18:18:16.669543 8009 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.669629 7993 ts_tablet_manager.cc:1397] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 18:18:16.669652 8009 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:18:16.669925 7993 raft_consensus.cc:348] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.669992 7993 raft_consensus.cc:374] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.670011 7993 raft_consensus.cc:729] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.670127 7993 consensus_queue.cc:260] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.670392 7993 ts_tablet_manager.cc:1428] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.670339 8008 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.670476 8008 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 18:18:16.670765 7721 tablet.cc:2369] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:16.672667 7994 tablet_bootstrap.cc:492] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: No bootstrap required, opened a new log
I20260808 18:18:16.672766 7994 ts_tablet_manager.cc:1397] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 18:18:16.673223 7994 raft_consensus.cc:348] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.673363 7994 raft_consensus.cc:374] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.673408 7994 raft_consensus.cc:729] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2735609afa4a7db97766090f039949, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.673535 7994 consensus_queue.cc:260] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.673810 7994 ts_tablet_manager.cc:1428] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:18:16.673759 8011 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.673888 8011 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:18:16.789012 8011 raft_consensus.cc:482] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:16.789147 8011 raft_consensus.cc:504] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.790346 8011 leader_election.cc:288] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569), 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633)
I20260808 18:18:16.794520 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.791170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:36165 (local address 127.27.18.194:41633)
0808 18:18:16.791401 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.791405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.791442 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:18:16.791501 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.791504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.791578 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.791665 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.792025 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.792539 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.793598 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.793954 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.794059 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.794113 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.794319 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.794394 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 18:18:16.794528 7778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.790591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33569 (local address 127.27.18.195:52721)
0808 18:18:16.790762 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.790778 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.790901 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.791189 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.791192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.791212 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.791929 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.791939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.792636 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.792639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.793482 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.793491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.794107 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.794110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.794239 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.794254 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.794351 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.794412 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":50}
I20260808 18:18:16.794555 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.790776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:52721 (local address 127.27.18.193:33569)
0808 18:18:16.790939 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.790945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.790963 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:18:16.791027 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.791033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.791088 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.791170 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:16.792061 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.792527 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.793627 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.793976 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.794098 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.794181 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.794383 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.794458 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 18:18:16.794534 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.790918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:41633 (local address 127.27.18.195:36165)
0808 18:18:16.791260 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.791273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.791385 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.791728 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.791731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.791751 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:16.791905 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.791911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.792670 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.792673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.793473 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.793481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.794069 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.794072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.794189 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.794216 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.794280 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.794365 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":121,"threads_started":1}
I20260808 18:18:16.795020 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d865dacafb44649c9aa7d73febf2db" candidate_uuid: "4c2735609afa4a7db97766090f039949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" is_pre_election: true
I20260808 18:18:16.795017 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d865dacafb44649c9aa7d73febf2db" candidate_uuid: "4c2735609afa4a7db97766090f039949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97" is_pre_election: true
I20260808 18:18:16.795163 7656 raft_consensus.cc:2454] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2735609afa4a7db97766090f039949 in term 0.
I20260808 18:18:16.795163 7434 raft_consensus.cc:2454] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2735609afa4a7db97766090f039949 in term 0.
I20260808 18:18:16.795534 7769 leader_election.cc:302] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, 4c2735609afa4a7db97766090f039949; no voters:
I20260808 18:18:16.795786 8011 raft_consensus.cc:2781] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:16.795853 8011 raft_consensus.cc:482] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:16.795881 8011 raft_consensus.cc:3037] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.796855 8011 raft_consensus.cc:504] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.797163 8011 leader_election.cc:288] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [CANDIDATE]: Term 1 election: Requested vote from peers d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569), 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633)
I20260808 18:18:16.797559 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d865dacafb44649c9aa7d73febf2db" candidate_uuid: "4c2735609afa4a7db97766090f039949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d99546e0bb7a432b9b2dab5485ab6edf"
I20260808 18:18:16.797559 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d865dacafb44649c9aa7d73febf2db" candidate_uuid: "4c2735609afa4a7db97766090f039949" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97"
I20260808 18:18:16.797677 7656 raft_consensus.cc:3037] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.797677 7434 raft_consensus.cc:3037] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:16.798777 7434 raft_consensus.cc:2454] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2735609afa4a7db97766090f039949 in term 1.
I20260808 18:18:16.798777 7656 raft_consensus.cc:2454] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2735609afa4a7db97766090f039949 in term 1.
I20260808 18:18:16.799090 7769 leader_election.cc:302] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, 4c2735609afa4a7db97766090f039949; no voters:
I20260808 18:18:16.799285 8011 raft_consensus.cc:2781] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:16.799500 8011 raft_consensus.cc:686] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [term 1 LEADER]: Becoming Leader. State: Replica: 4c2735609afa4a7db97766090f039949, State: Running, Role: LEADER
I20260808 18:18:16.799736 8011 consensus_queue.cc:237] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.800808 8068 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: TxnStatusTablet state changed. Reason: New leader 4c2735609afa4a7db97766090f039949. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.800921 8068 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:18:16.801200 8071 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 18:18:16.801353 8071 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 18:18:16.801963 7188 catalog_manager.cc:5495] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2735609afa4a7db97766090f039949 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } health_report { overall_health: HEALTHY } } }
I20260808 18:18:16.829731 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.825959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.27.18.195:35313)
0808 18:18:16.826080 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.826083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.826174 (+ 91us) server_negotiation.cc:407] Connection header received
0808 18:18:16.826349 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.826351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.826398 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.826476 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:16.826724 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.827227 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.828429 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.828642 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.828720 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.829045 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.829405 (+ 360us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.829481 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.829582 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.829624 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:18:16.829794 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.825876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35313 (local address 127.0.0.1:55346)
0808 18:18:16.826100 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.826114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.826219 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.826511 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.826514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.826522 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:18:16.826626 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.826632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.827330 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.827333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.828322 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.828333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.828744 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.828748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.828851 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.828900 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.829487 (+ 587us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.829503 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.829571 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.829650 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 18:18:16.832459 7434 raft_consensus.cc:1261] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Refusing update from remote peer 4c2735609afa4a7db97766090f039949: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:16.832558 7656 raft_consensus.cc:1261] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Refusing update from remote peer 4c2735609afa4a7db97766090f039949: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:16.832911 8011 consensus_queue.cc:1035] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:16.833127 8068 consensus_queue.cc:1035] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:16.835672 8008 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: TxnStatusTablet state changed. Reason: New leader 4c2735609afa4a7db97766090f039949. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.835662 8009 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: TxnStatusTablet state changed. Reason: New leader 4c2735609afa4a7db97766090f039949. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.835767 8009 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:18:16.835767 8008 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:18:16.837198 8011 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.837317 8011 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:18:16.837622 8009 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.837707 8009 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 08025cae082f46c2b28f078311a09f97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:18:16.839363 8011 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.839512 8011 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P 4c2735609afa4a7db97766090f039949: This TxnStatusTablet replica's current role is: LEADER
I20260808 18:18:16.840622 8018 mvcc.cc:205] Tried to move back new op lower bound from 7316328844620709888 to 7316328844493832192. Current Snapshot: MvccSnapshot[applied={T|T < 7316328844620709888}]
I20260808 18:18:16.840817 8008 tablet_replica.cc:439] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c2735609afa4a7db97766090f039949" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } } }
I20260808 18:18:16.840921 8008 tablet_replica.cc:442] T 75d865dacafb44649c9aa7d73febf2db P d99546e0bb7a432b9b2dab5485ab6edf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 18:18:16.846695 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.843130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55420 (local address 127.27.18.193:33569)
0808 18:18:16.843250 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.843254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.843268 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:16.843421 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.843424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.843484 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.843582 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:16.843814 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.844600 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.845373 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.845601 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.845706 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.845967 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.846321 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.846439 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.846502 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.846555 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 18:18:16.856882 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.853314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:35437 (local address 127.27.18.195:35313)
0808 18:18:16.853438 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.853442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.853628 (+ 186us) server_negotiation.cc:407] Connection header received
0808 18:18:16.853755 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.853757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.853792 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.853856 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:16.854144 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.854672 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.855855 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.856055 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.856130 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.856445 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.856501 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.856552 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.856708 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.856748 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:18:16.857199 8081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.853253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35313 (local address 127.27.18.193:35437)
0808 18:18:16.853542 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.853558 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.853667 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.853890 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.853893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.853901 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:18:16.854022 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.854029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.854765 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.854769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.855736 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.855750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.856162 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.856167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.856267 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.856320 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.856600 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.856607 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.856699 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.856776 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260808 18:18:16.864063 7656 raft_consensus.cc:1261] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Refusing update from remote peer d99546e0bb7a432b9b2dab5485ab6edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:16.864122 7876 raft_consensus.cc:1261] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Refusing update from remote peer d99546e0bb7a432b9b2dab5485ab6edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:16.864591 8045 consensus_queue.cc:1035] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:16.864730 8008 consensus_queue.cc:1035] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:16.869839 8017 mvcc.cc:205] Tried to move back new op lower bound from 7316328844751175680 to 7316328843731406848. Current Snapshot: MvccSnapshot[applied={T|T < 7316328844751175680}]
I20260808 18:18:16.887990 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.884181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33569 (local address 127.27.18.195:34625)
0808 18:18:16.884569 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:16.884586 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:16.884685 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:16.884919 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:16.884922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:16.884930 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 18:18:16.885051 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.885056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.885778 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.885781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.886617 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:16.886627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.887290 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.887293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:16.887420 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.887474 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.887772 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.887775 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 18:18:16.887815 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 18:18:16.887874 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":74,"threads_started":1}
I20260808 18:18:16.888022 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.884518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:34625 (local address 127.27.18.193:33569)
0808 18:18:16.884652 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.884657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.884670 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:16.884779 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.884781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.884822 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.884904 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:16.885155 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.885652 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.886788 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.887133 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.887250 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.887600 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:16.887685 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:16.887776 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.887840 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.887892 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:18:16.910985 7188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:16.911146 7188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:16.913319 7188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Sending DeleteTablet for 3 replicas of tablet b6656f51020b40ec8545b8a04b5447c5
I20260808 18:18:16.913899 7623 tablet_service.cc:1514] Processing DeleteTablet for tablet b6656f51020b40ec8545b8a04b5447c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:16 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33818
I20260808 18:18:16.914085 8036 tablet_replica.cc:330] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: stopping tablet replica
I20260808 18:18:16.914217 8036 raft_consensus.cc:2229] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:16.914323 8036 raft_consensus.cc:2258] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:16.914738 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet b6656f51020b40ec8545b8a04b5447c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:16 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55412
I20260808 18:18:16.914915 8037 tablet_replica.cc:330] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: stopping tablet replica
I20260808 18:18:16.914914 7842 tablet_service.cc:1514] Processing DeleteTablet for tablet b6656f51020b40ec8545b8a04b5447c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:16 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55334
I20260808 18:18:16.915051 8037 raft_consensus.cc:2229] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:16.915083 8035 tablet_replica.cc:330] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: stopping tablet replica
I20260808 18:18:16.915220 8035 raft_consensus.cc:2229] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:16.915329 8035 raft_consensus.cc:2258] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:16.915330 8037 raft_consensus.cc:2258] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:16.916109 8037 ts_tablet_manager.cc:1905] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:16.916231 8036 ts_tablet_manager.cc:1905] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:16.918972 8037 ts_tablet_manager.cc:1918] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:18:16.919039 8037 log.cc:1196] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/b6656f51020b40ec8545b8a04b5447c5
I20260808 18:18:16.919157 8036 ts_tablet_manager.cc:1918] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:18:16.919234 8036 log.cc:1196] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/b6656f51020b40ec8545b8a04b5447c5
I20260808 18:18:16.919296 8037 ts_tablet_manager.cc:1939] T b6656f51020b40ec8545b8a04b5447c5 P d99546e0bb7a432b9b2dab5485ab6edf: Deleting consensus metadata
I20260808 18:18:16.919579 8036 ts_tablet_manager.cc:1939] T b6656f51020b40ec8545b8a04b5447c5 P 08025cae082f46c2b28f078311a09f97: Deleting consensus metadata
I20260808 18:18:16.920069 7160 catalog_manager.cc:4841] TS d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569): tablet b6656f51020b40ec8545b8a04b5447c5 (table test-table [id=2a086061b64245989135a7bc65b23702]) successfully deleted
I20260808 18:18:16.920977 8035 ts_tablet_manager.cc:1905] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:16.924242 8035 ts_tablet_manager.cc:1918] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:18:16.924351 8035 log.cc:1196] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/b6656f51020b40ec8545b8a04b5447c5
I20260808 18:18:16.924633 8035 ts_tablet_manager.cc:1939] T b6656f51020b40ec8545b8a04b5447c5 P 4c2735609afa4a7db97766090f039949: Deleting consensus metadata
I20260808 18:18:16.925333 7164 catalog_manager.cc:4841] TS 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313): tablet b6656f51020b40ec8545b8a04b5447c5 (table test-table [id=2a086061b64245989135a7bc65b23702]) successfully deleted
I20260808 18:18:16.927595 7160 catalog_manager.cc:4841] TS 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633): tablet b6656f51020b40ec8545b8a04b5447c5 (table test-table [id=2a086061b64245989135a7bc65b23702]) successfully deleted
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:18:16.956845 7261 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:16.948594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.27.18.254:35849)
0808 18:18:16.948733 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.948739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.948960 (+ 221us) server_negotiation.cc:407] Connection header received
0808 18:18:16.949106 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.949110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.949171 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.949282 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.950155 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.950925 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.952172 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.952478 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.952568 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.956720 (+ 4152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59504: BlockingRecv error: recv got EOF from 127.0.0.1:59504 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 18:18:16 dist-test-slave-x4qh krb5kdc[7113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM
I20260808 18:18:16.974128 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:16.959739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.27.18.254:35849)
0808 18:18:16.959842 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:16.959846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:16.963637 (+ 3791us) server_negotiation.cc:407] Connection header received
0808 18:18:16.963811 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:16.963814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:16.963857 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:16.963908 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:16.964778 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.965530 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.966673 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:16.966976 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:16.967044 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:16.969896 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:16.969900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:16.969903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:16.969942 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:16.971707 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.972560 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.972565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.972567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.972743 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:16.973182 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:16.973186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:16.973188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:16.973407 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:16.973519 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:16.973843 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 18:18:16.973951 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:18:16.977505 7188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:16.977952 7188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:16.983011 7842 tablet_service.cc:1467] Processing CreateTablet for tablet df7d4ab8b88c470587d6d48c4d451a7f (DEFAULT_TABLE table=test-table [id=db3b47dbd5504568a608032100f275b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.983079 7400 tablet_service.cc:1467] Processing CreateTablet for tablet df7d4ab8b88c470587d6d48c4d451a7f (DEFAULT_TABLE table=test-table [id=db3b47dbd5504568a608032100f275b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.983320 7842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7d4ab8b88c470587d6d48c4d451a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.983377 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7d4ab8b88c470587d6d48c4d451a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.985790 7994 tablet_bootstrap.cc:492] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Bootstrap starting.
I20260808 18:18:16.985900 7623 tablet_service.cc:1467] Processing CreateTablet for tablet df7d4ab8b88c470587d6d48c4d451a7f (DEFAULT_TABLE table=test-table [id=db3b47dbd5504568a608032100f275b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:16.986155 7623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7d4ab8b88c470587d6d48c4d451a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:16.986578 7994 tablet_bootstrap.cc:654] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.988505 7997 tablet_bootstrap.cc:492] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Bootstrap starting.
I20260808 18:18:16.988547 7993 tablet_bootstrap.cc:492] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Bootstrap starting.
I20260808 18:18:16.989357 7997 tablet_bootstrap.cc:654] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.989429 7993 tablet_bootstrap.cc:654] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:16.990618 7993 tablet_bootstrap.cc:492] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: No bootstrap required, opened a new log
I20260808 18:18:16.990700 7993 ts_tablet_manager.cc:1397] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:16.991093 7993 raft_consensus.cc:348] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.991230 7993 raft_consensus.cc:374] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.991278 7993 raft_consensus.cc:729] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.991452 7993 consensus_queue.cc:260] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.991717 7993 ts_tablet_manager.cc:1428] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.992223 7997 tablet_bootstrap.cc:492] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: No bootstrap required, opened a new log
I20260808 18:18:16.992306 7997 ts_tablet_manager.cc:1397] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 18:18:16.992657 7997 raft_consensus.cc:348] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.992776 7997 raft_consensus.cc:374] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.992822 7997 raft_consensus.cc:729] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08025cae082f46c2b28f078311a09f97, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.992957 7997 consensus_queue.cc:260] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.993255 7997 ts_tablet_manager.cc:1428] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:16.993844 7994 tablet_bootstrap.cc:492] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: No bootstrap required, opened a new log
I20260808 18:18:16.993933 7994 ts_tablet_manager.cc:1397] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 18:18:16.994360 7994 raft_consensus.cc:348] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.994467 7994 raft_consensus.cc:374] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:16.994490 7994 raft_consensus.cc:729] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2735609afa4a7db97766090f039949, State: Initialized, Role: FOLLOWER
I20260808 18:18:16.994622 7994 consensus_queue.cc:260] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:16.994874 7994 ts_tablet_manager.cc:1428] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:18:17.179173 8089 raft_consensus.cc:482] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:17.179317 8089 raft_consensus.cc:504] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.179755 8089 leader_election.cc:288] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633), 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:17.180119 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7d4ab8b88c470587d6d48c4d451a7f" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97" is_pre_election: true
I20260808 18:18:17.180127 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7d4ab8b88c470587d6d48c4d451a7f" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949" is_pre_election: true
I20260808 18:18:17.180255 7656 raft_consensus.cc:2454] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 0.
I20260808 18:18:17.180281 7876 raft_consensus.cc:2454] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 0.
I20260808 18:18:17.180517 7325 leader_election.cc:302] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:17.180702 8089 raft_consensus.cc:2781] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:17.180764 8089 raft_consensus.cc:482] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:17.180795 8089 raft_consensus.cc:3037] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:17.181926 8089 raft_consensus.cc:504] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.182235 8089 leader_election.cc:288] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 election: Requested vote from peers 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633), 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:17.182519 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7d4ab8b88c470587d6d48c4d451a7f" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97"
I20260808 18:18:17.182643 7656 raft_consensus.cc:3037] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:17.182651 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7d4ab8b88c470587d6d48c4d451a7f" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949"
I20260808 18:18:17.182760 7876 raft_consensus.cc:3037] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:17.183789 7656 raft_consensus.cc:2454] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 1.
I20260808 18:18:17.183789 7876 raft_consensus.cc:2454] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 1.
I20260808 18:18:17.184041 7327 leader_election.cc:302] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2735609afa4a7db97766090f039949, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:17.184203 8089 raft_consensus.cc:2781] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:17.184293 8089 raft_consensus.cc:686] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 1 LEADER]: Becoming Leader. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Running, Role: LEADER
I20260808 18:18:17.184442 8089 consensus_queue.cc:237] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.186187 7187 catalog_manager.cc:5495] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf reported cstate change: term changed from 0 to 1, leader changed from <none> to d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193). New cstate: current_term: 1 leader_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } health_report { overall_health: UNKNOWN } } }
I20260808 18:18:17.227780 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:17.225108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.27.18.193:33569)
0808 18:18:17.225218 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:17.225221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:17.225233 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:17.225325 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:17.225327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:17.225366 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:17.225427 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:17.225651 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:17.226151 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:17.226906 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:17.227091 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:17.227159 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:17.227419 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:17.227486 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:17.227539 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:17.227649 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 18:18:17.227689 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:18:17.229681 7876 raft_consensus.cc:1261] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Refusing update from remote peer d99546e0bb7a432b9b2dab5485ab6edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:17.229693 7656 raft_consensus.cc:1261] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Refusing update from remote peer d99546e0bb7a432b9b2dab5485ab6edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:17.230137 8088 consensus_queue.cc:1035] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:17.230341 8089 consensus_queue.cc:1035] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:17.238014 7187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:17.238114 7187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:17.239749 7187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Sending DeleteTablet for 3 replicas of tablet df7d4ab8b88c470587d6d48c4d451a7f
I20260808 18:18:17.240244 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet df7d4ab8b88c470587d6d48c4d451a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:17 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55412
I20260808 18:18:17.240305 7623 tablet_service.cc:1514] Processing DeleteTablet for tablet df7d4ab8b88c470587d6d48c4d451a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:17 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33818
I20260808 18:18:17.240418 8037 tablet_replica.cc:330] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: stopping tablet replica
I20260808 18:18:17.240412 7842 tablet_service.cc:1514] Processing DeleteTablet for tablet df7d4ab8b88c470587d6d48c4d451a7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:17 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55334
I20260808 18:18:17.240483 8036 tablet_replica.cc:330] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: stopping tablet replica
I20260808 18:18:17.240525 8037 raft_consensus.cc:2229] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:17.240554 8035 tablet_replica.cc:330] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: stopping tablet replica
I20260808 18:18:17.240587 8036 raft_consensus.cc:2229] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:17.240635 8035 raft_consensus.cc:2229] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:17.240674 8037 raft_consensus.cc:2258] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:17.240692 8036 raft_consensus.cc:2258] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:17.240732 8035 raft_consensus.cc:2258] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:17.241411 8037 ts_tablet_manager.cc:1905] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:17.241417 8036 ts_tablet_manager.cc:1905] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:17.243148 8037 ts_tablet_manager.cc:1918] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:17.243235 8037 log.cc:1196] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/df7d4ab8b88c470587d6d48c4d451a7f
I20260808 18:18:17.243279 8036 ts_tablet_manager.cc:1918] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:17.243407 8036 log.cc:1196] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/df7d4ab8b88c470587d6d48c4d451a7f
I20260808 18:18:17.243530 8037 ts_tablet_manager.cc:1939] T df7d4ab8b88c470587d6d48c4d451a7f P d99546e0bb7a432b9b2dab5485ab6edf: Deleting consensus metadata
I20260808 18:18:17.243728 8036 ts_tablet_manager.cc:1939] T df7d4ab8b88c470587d6d48c4d451a7f P 08025cae082f46c2b28f078311a09f97: Deleting consensus metadata
I20260808 18:18:17.244180 7160 catalog_manager.cc:4841] TS d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569): tablet df7d4ab8b88c470587d6d48c4d451a7f (table test-table [id=db3b47dbd5504568a608032100f275b7]) successfully deleted
I20260808 18:18:17.244220 7187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 18:18:17.244357 7160 catalog_manager.cc:4841] TS 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633): tablet df7d4ab8b88c470587d6d48c4d451a7f (table test-table [id=db3b47dbd5504568a608032100f275b7]) successfully deleted
W20260808 18:18:17.244627 7187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:17.245579 8035 ts_tablet_manager.cc:1905] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:17.247777 8035 ts_tablet_manager.cc:1918] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:17.247872 8035 log.cc:1196] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/df7d4ab8b88c470587d6d48c4d451a7f
I20260808 18:18:17.248134 8035 ts_tablet_manager.cc:1939] T df7d4ab8b88c470587d6d48c4d451a7f P 4c2735609afa4a7db97766090f039949: Deleting consensus metadata
I20260808 18:18:17.248703 7164 catalog_manager.cc:4841] TS 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313): tablet df7d4ab8b88c470587d6d48c4d451a7f (table test-table [id=db3b47dbd5504568a608032100f275b7]) successfully deleted
I20260808 18:18:17.248878 7400 tablet_service.cc:1467] Processing CreateTablet for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e (DEFAULT_TABLE table=test-table [id=ee640d95391f4a698b09023b0c563e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:17.248957 7623 tablet_service.cc:1467] Processing CreateTablet for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e (DEFAULT_TABLE table=test-table [id=ee640d95391f4a698b09023b0c563e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:17.249104 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:17.249194 7623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:17.249207 7842 tablet_service.cc:1467] Processing CreateTablet for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e (DEFAULT_TABLE table=test-table [id=ee640d95391f4a698b09023b0c563e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:17.249428 7842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:17.250581 7993 tablet_bootstrap.cc:492] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf: Bootstrap starting.
I20260808 18:18:17.251502 7994 tablet_bootstrap.cc:492] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Bootstrap starting.
I20260808 18:18:17.251556 7993 tablet_bootstrap.cc:654] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:17.252007 7997 tablet_bootstrap.cc:492] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Bootstrap starting.
I20260808 18:18:17.252211 7994 tablet_bootstrap.cc:654] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:17.252766 7997 tablet_bootstrap.cc:654] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:17.253331 7994 tablet_bootstrap.cc:492] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: No bootstrap required, opened a new log
I20260808 18:18:17.253436 7994 ts_tablet_manager.cc:1397] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:17.253654 7993 tablet_bootstrap.cc:492] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf: No bootstrap required, opened a new log
I20260808 18:18:17.253739 7993 ts_tablet_manager.cc:1397] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:18:17.253875 7994 raft_consensus.cc:348] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.253958 7994 raft_consensus.cc:374] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:17.254019 7994 raft_consensus.cc:729] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2735609afa4a7db97766090f039949, State: Initialized, Role: FOLLOWER
I20260808 18:18:17.254112 7993 raft_consensus.cc:348] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.254136 7994 consensus_queue.cc:260] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.254216 7993 raft_consensus.cc:374] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:17.254256 7993 raft_consensus.cc:729] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Initialized, Role: FOLLOWER
I20260808 18:18:17.254354 7994 ts_tablet_manager.cc:1428] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:18:17.254374 7993 consensus_queue.cc:260] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.254655 7993 ts_tablet_manager.cc:1428] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:17.255205 7997 tablet_bootstrap.cc:492] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: No bootstrap required, opened a new log
I20260808 18:18:17.255288 7997 ts_tablet_manager.cc:1397] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 18:18:17.255687 7997 raft_consensus.cc:348] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.255788 7997 raft_consensus.cc:374] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:17.255832 7997 raft_consensus.cc:729] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08025cae082f46c2b28f078311a09f97, State: Initialized, Role: FOLLOWER
I20260808 18:18:17.255961 7997 consensus_queue.cc:260] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.256211 7997 ts_tablet_manager.cc:1428] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:18:17.256448 8089 raft_consensus.cc:482] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:17.256532 8089 raft_consensus.cc:504] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.256927 8089 leader_election.cc:288] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633), 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:17.257151 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5e68041fc9c4a7f9ef8a892a23f6f0e" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97" is_pre_election: true
I20260808 18:18:17.257282 7656 raft_consensus.cc:2454] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 0.
I20260808 18:18:17.257580 7325 leader_election.cc:302] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:17.257735 8089 raft_consensus.cc:2781] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:17.257781 8089 raft_consensus.cc:482] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:17.257828 8089 raft_consensus.cc:3037] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:17.257963 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5e68041fc9c4a7f9ef8a892a23f6f0e" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949" is_pre_election: true
I20260808 18:18:17.258083 7876 raft_consensus.cc:2454] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 0.
I20260808 18:18:17.258733 8089 raft_consensus.cc:504] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.259059 8089 leader_election.cc:288] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 election: Requested vote from peers 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633), 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313)
I20260808 18:18:17.259341 7656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5e68041fc9c4a7f9ef8a892a23f6f0e" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08025cae082f46c2b28f078311a09f97"
I20260808 18:18:17.259486 7656 raft_consensus.cc:3037] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:17.259469 7876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5e68041fc9c4a7f9ef8a892a23f6f0e" candidate_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2735609afa4a7db97766090f039949"
I20260808 18:18:17.259563 7876 raft_consensus.cc:3037] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:17.260198 7656 raft_consensus.cc:2454] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 1.
I20260808 18:18:17.260243 7876 raft_consensus.cc:2454] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d99546e0bb7a432b9b2dab5485ab6edf in term 1.
I20260808 18:18:17.260443 7325 leader_election.cc:302] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08025cae082f46c2b28f078311a09f97, d99546e0bb7a432b9b2dab5485ab6edf; no voters:
I20260808 18:18:17.260597 8089 raft_consensus.cc:2781] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:17.260676 8089 raft_consensus.cc:686] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 1 LEADER]: Becoming Leader. State: Replica: d99546e0bb7a432b9b2dab5485ab6edf, State: Running, Role: LEADER
I20260808 18:18:17.260779 8089 consensus_queue.cc:237] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } }
I20260808 18:18:17.262207 7186 catalog_manager.cc:5495] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf reported cstate change: term changed from 0 to 1, leader changed from <none> to d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193). New cstate: current_term: 1 leader_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d99546e0bb7a432b9b2dab5485ab6edf" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 } health_report { overall_health: UNKNOWN } } }
I20260808 18:18:17.277920 7876 raft_consensus.cc:1261] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Refusing update from remote peer d99546e0bb7a432b9b2dab5485ab6edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:17.277938 7656 raft_consensus.cc:1261] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Refusing update from remote peer d99546e0bb7a432b9b2dab5485ab6edf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:17.278246 8089 consensus_queue.cc:1035] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2735609afa4a7db97766090f039949" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:17.278421 8087 consensus_queue.cc:1035] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [LEADER]: Connected to new peer: Peer: permanent_uuid: "08025cae082f46c2b28f078311a09f97" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:17.303062 7186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:17.303225 7186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:17.304947 7186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9198006a60944565ae15fd1b4d505f9e: Sending DeleteTablet for 3 replicas of tablet c5e68041fc9c4a7f9ef8a892a23f6f0e
I20260808 18:18:17.305430 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:17 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55412
I20260808 18:18:17.305487 7623 tablet_service.cc:1514] Processing DeleteTablet for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:17 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33818
I20260808 18:18:17.305620 8037 tablet_replica.cc:330] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf: stopping tablet replica
I20260808 18:18:17.305672 7842 tablet_service.cc:1514] Processing DeleteTablet for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:17 UTC) from {username='oryx', principal='oryx/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55334
I20260808 18:18:17.305743 8037 raft_consensus.cc:2229] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:17.305824 8035 tablet_replica.cc:330] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: stopping tablet replica
I20260808 18:18:17.305929 8037 raft_consensus.cc:2258] T c5e68041fc9c4a7f9ef8a892a23f6f0e P d99546e0bb7a432b9b2dab5485ab6edf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:17.305933 8035 raft_consensus.cc:2229] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:17.306037 8035 raft_consensus.cc:2258] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:17.306061 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 7295
I20260808 18:18:17.312021 8036 tablet_replica.cc:330] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: stopping tablet replica
I20260808 18:18:17.312181 8036 raft_consensus.cc:2229] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:17.312305 8036 raft_consensus.cc:2258] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:17.313485 8036 ts_tablet_manager.cc:1905] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:17.316044 8035 ts_tablet_manager.cc:1905] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:17.316679 8036 ts_tablet_manager.cc:1918] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:18:17.316785 8036 log.cc:1196] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/c5e68041fc9c4a7f9ef8a892a23f6f0e
I20260808 18:18:17.317006 8036 ts_tablet_manager.cc:1939] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 08025cae082f46c2b28f078311a09f97: Deleting consensus metadata
I20260808 18:18:17.317734 7160 catalog_manager.cc:4841] TS 08025cae082f46c2b28f078311a09f97 (127.27.18.194:41633): tablet c5e68041fc9c4a7f9ef8a892a23f6f0e (table test-table [id=ee640d95391f4a698b09023b0c563e87]) successfully deleted
I20260808 18:18:17.318234 8035 ts_tablet_manager.cc:1918] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 18:18:17.318324 8035 log.cc:1196] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/c5e68041fc9c4a7f9ef8a892a23f6f0e
I20260808 18:18:17.318583 8035 ts_tablet_manager.cc:1939] T c5e68041fc9c4a7f9ef8a892a23f6f0e P 4c2735609afa4a7db97766090f039949: Deleting consensus metadata
I20260808 18:18:17.319155 7164 catalog_manager.cc:4841] TS 4c2735609afa4a7db97766090f039949 (127.27.18.195:35313): tablet c5e68041fc9c4a7f9ef8a892a23f6f0e (table test-table [id=ee640d95391f4a698b09023b0c563e87]) successfully deleted
I20260808 18:18:17.321772 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 7518
W20260808 18:18:17.321758 7160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:33569 (error 108)
I20260808 18:18:17.327069 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:17.326545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33569 (local address 127.0.0.1:55440)
0808 18:18:17.326860 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:17.326945 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":70,"threads_started":1}
W20260808 18:18:17.327207 7160 catalog_manager.cc:4584] TS d99546e0bb7a432b9b2dab5485ab6edf (127.27.18.193:33569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c5e68041fc9c4a7f9ef8a892a23f6f0e: Network error: Client connection negotiation failed: client connection to 127.27.18.193:33569: connect: Connection refused (error 111)
I20260808 18:18:17.336894 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 7736
I20260808 18:18:17.348513 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 7136
2026-08-08T18:18:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3125 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:17 dist-test-slave-x4qh krb5kdc[8142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:17 dist-test-slave-x4qh krb5kdc[8142](info): set up 2 sockets
Aug 08 18:18:17 dist-test-slave-x4qh krb5kdc[8142](info): commencing operation
krb5kdc: starting...
W20260808 18:18:18.527597 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:18 dist-test-slave-x4qh krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:18:18.639951 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:37667
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:37667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:18.746238 8173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:18.746464 8173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:18.746501 8173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:18.750264 8173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:18.750320 8173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:18.750340 8173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:18.750356 8173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:18.750393 8173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:18.755216 8173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:37667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:18.756114 8173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:18.757026 8173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:18.765004 8186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:18.765058 8184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:18.765000 8189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:18.765116 8173 server_base.cc:1034] running on GCE node
I20260808 18:18:18.765838 8173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:18.766808 8173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:18.767961 8173 hybrid_clock.cc:648] HybridClock initialized: now 1786213098767946 us; error 25 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T18:18:18Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1424 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:18 dist-test-slave-x4qh krb5kdc[8194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:18 dist-test-slave-x4qh krb5kdc[8194](info): set up 2 sockets
Aug 08 18:18:18 dist-test-slave-x4qh krb5kdc[8194](info): commencing operation
krb5kdc: starting...
W20260808 18:18:19.947299 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:18:20.058975 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:34043
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:34043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:20.163946 8212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:20.164192 8212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:20.164268 8212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:20.168077 8212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:20.168146 8212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:20.168185 8212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:20.168257 8212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:20.168291 8212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:20.173071 8212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:34043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:34043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:20.173949 8212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:20.174844 8212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:20.182533 8225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.182551 8220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.182548 8221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:20.182746 8212 server_base.cc:1034] running on GCE node
I20260808 18:18:20.183241 8212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:20.184190 8212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:20.185369 8212 hybrid_clock.cc:648] HybridClock initialized: now 1786213100185349 us; error 30 us; skew 500 ppm
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:20.187994 8212 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:18:20.188787 8212 webserver.cc:460] Webserver started at http://127.27.18.254:42041/ using document root <none> and password file <none>
I20260808 18:18:20.189253 8212 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:20.189344 8212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:20.189535 8212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:20.191255 8212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "50bb831405d645d4840013d99911b345"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "45ac3ede496d410a3350e8b6b3018b63"
server_key_iv: "8662324fb9991c0558a91c53410a830f"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.191871 8212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "50bb831405d645d4840013d99911b345"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "45ac3ede496d410a3350e8b6b3018b63"
server_key_iv: "8662324fb9991c0558a91c53410a830f"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.195111 8212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:20.197284 8234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.198297 8212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:18:20.198422 8212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "50bb831405d645d4840013d99911b345"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "45ac3ede496d410a3350e8b6b3018b63"
server_key_iv: "8662324fb9991c0558a91c53410a830f"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.198508 8212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:20.220902 8212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:20.223255 8212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:20.223456 8212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:20.230000 8212 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:34043
I20260808 18:18:20.230017 8329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:34043 every 8 connection(s)
I20260808 18:18:20.230803 8212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:20.233227 8330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:20.234242 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 8212
I20260808 18:18:20.234385 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:20.234632 27723 external_mini_cluster.cc:1428] Setting key 6f8614f463476b20197ac29c992ba149
I20260808 18:18:20.239223 8330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Bootstrap starting.
I20260808 18:18:20.241501 8330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:20.242180 8330 log.cc:824] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:20.244222 8330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: No bootstrap required, opened a new log
I20260808 18:18:20.247849 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.235942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.27.18.254:34043)
0808 18:18:20.236348 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:20.236366 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:20.236406 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:18:20.237200 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:20.237220 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:20.237586 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:20.237899 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:20.238473 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.239808 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.240575 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.241173 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.241325 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.243676 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:20.243681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:20.243692 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:20.243716 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:20.245289 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.245753 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.245774 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.245778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.245879 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.246179 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.246181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.246182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.246517 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:20.246647 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:20.246920 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 18:18:20.247183 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":99,"threads_started":1}
I20260808 18:18:20.248049 8330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:20.248296 8330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:20.248369 8330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bb831405d645d4840013d99911b345, State: Initialized, Role: FOLLOWER
I20260808 18:18:20.248756 8330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:20.248894 8330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:20.248970 8330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:20.249058 8330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:20.249789 8330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:20.250102 8330 leader_election.cc:302] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50bb831405d645d4840013d99911b345; no voters:
I20260808 18:18:20.250316 8330 leader_election.cc:288] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:20.250425 8337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:20.250634 8337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 LEADER]: Becoming Leader. State: Replica: 50bb831405d645d4840013d99911b345, State: Running, Role: LEADER
I20260808 18:18:20.250900 8337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:20.251310 8330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:20.252427 8340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50bb831405d645d4840013d99911b345. Latest consensus state: current_term: 1 leader_uuid: "50bb831405d645d4840013d99911b345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } } }
I20260808 18:18:20.252454 8339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50bb831405d645d4840013d99911b345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } } }
I20260808 18:18:20.252565 8340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:20.252578 8339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:20.252923 8352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:20.255714 8352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:20.260731 8352 catalog_manager.cc:1347] Generated new cluster ID: 0b3d4b78434b43c0b1e5016b8f24eb53
I20260808 18:18:20.260792 8352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:20.285279 8352 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:20.286090 8352 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:20.292971 8352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Generated new TSK 0
I20260808 18:18:20.293560 8352 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:18:20.325888 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34043
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:20.457479 8375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:20.457690 8375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:20.457741 8375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:20.461169 8375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:20.461206 8375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:20.461252 8375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:20.465929 8375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:20.466773 8375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:20.467649 8375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:20.476583 8384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.476660 8385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.476770 8390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:20.476908 8375 server_base.cc:1034] running on GCE node
I20260808 18:18:20.477298 8375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:20.477828 8375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:20.479001 8375 hybrid_clock.cc:648] HybridClock initialized: now 1786213100478988 us; error 36 us; skew 500 ppm
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:20.481640 8375 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:18:20.482417 8375 webserver.cc:460] Webserver started at http://127.27.18.193:44735/ using document root <none> and password file <none>
I20260808 18:18:20.482933 8375 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:20.483017 8375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:20.483253 8375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:20.485186 8375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "b39877d6e8e441928d26ac89eaf534b6"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "c364cee8d66acc07e4e63cccdb389d8d"
server_key_iv: "7ee26dd817e84fa920d2da9f02610986"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.485733 8375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "b39877d6e8e441928d26ac89eaf534b6"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "c364cee8d66acc07e4e63cccdb389d8d"
server_key_iv: "7ee26dd817e84fa920d2da9f02610986"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.489059 8375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:20.491183 8400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.492280 8375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:20.492403 8375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "b39877d6e8e441928d26ac89eaf534b6"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "c364cee8d66acc07e4e63cccdb389d8d"
server_key_iv: "7ee26dd817e84fa920d2da9f02610986"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.492501 8375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:20.505424 8375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:20.507953 8375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:20.508132 8375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:20.508697 8375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:20.509443 8375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:20.509511 8375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.509575 8375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:20.509624 8375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.518105 8375 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:32909
I20260808 18:18:20.518132 8582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:32909 every 8 connection(s)
I20260808 18:18:20.518860 8375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:20.522289 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 8375
I20260808 18:18:20.522433 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:20.522740 27723 external_mini_cluster.cc:1428] Setting key e94ee4c2fc40e62dcecc16e6f112b7a7
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:20.532868 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.520498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:55755 (local address 127.27.18.254:34043)
0808 18:18:20.520623 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:20.520627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:20.521377 (+ 750us) server_negotiation.cc:407] Connection header received
0808 18:18:20.522875 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:20.522878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:20.522924 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:20.522991 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:20.524285 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.524812 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.526483 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.526903 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.527022 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.529231 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:20.529237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:20.529240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:20.529268 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:20.530857 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.531295 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.531299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.531301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.531441 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.531748 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.531752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.531755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.532039 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:20.532142 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:20.532546 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 18:18:20.532671 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:18:20.533407 8588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.520824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:55755)
0808 18:18:20.521241 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:20.521278 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:20.522619 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:20.523172 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:20.523178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:20.523602 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:20.524136 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:20.524154 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.524937 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.524943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:20.526367 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:20.526377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.527041 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.527043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:20.527173 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.527711 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:20.527726 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:20.529064 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:20.530965 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:20.530969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:20.530978 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:20.531201 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:20.531528 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:20.531530 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:20.531532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:20.531654 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:20.532133 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:20.532137 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:20.532326 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 18:18:20.532457 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 18:18:20.532658 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260808 18:18:20.534400 8584 heartbeater.cc:347] Connected to a master server at 127.27.18.254:34043
I20260808 18:18:20.534607 8584 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:20.535013 8584 heartbeater.cc:511] Master 127.27.18.254:34043 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:18:20.536558 8261 ts_manager.cc:194] Registered new tserver with Master: b39877d6e8e441928d26ac89eaf534b6 (127.27.18.193:32909)
I20260808 18:18:20.537609 8261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:55755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:18:20.549328 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34043
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:20.653658 8591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:20.653884 8591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:20.653962 8591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:20.657720 8591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:20.657771 8591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:20.657826 8591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:20.662245 8591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:20.663002 8591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:20.663898 8591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:18:20.672516 8591 server_base.cc:1034] running on GCE node
W20260808 18:18:20.672459 8608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.672464 8602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.672472 8601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:20.673336 8591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:20.673874 8591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:20.675061 8591 hybrid_clock.cc:648] HybridClock initialized: now 1786213100675043 us; error 32 us; skew 500 ppm
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:20.677568 8591 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:18:20.678354 8591 webserver.cc:460] Webserver started at http://127.27.18.194:33129/ using document root <none> and password file <none>
I20260808 18:18:20.678866 8591 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:20.678951 8591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:20.679162 8591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:20.681241 8591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "2edf68af071c48fdb22278eb02120faf"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "1836604c6adf1faeae2255402abaabe0"
server_key_iv: "0c889b4ac4c3913b6480b6062e36979b"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.681792 8591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "2edf68af071c48fdb22278eb02120faf"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "1836604c6adf1faeae2255402abaabe0"
server_key_iv: "0c889b4ac4c3913b6480b6062e36979b"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.685281 8591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:20.687458 8616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.688449 8591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:20.688568 8591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "2edf68af071c48fdb22278eb02120faf"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "1836604c6adf1faeae2255402abaabe0"
server_key_iv: "0c889b4ac4c3913b6480b6062e36979b"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.688663 8591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:20.704602 8591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:20.707093 8591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:20.707273 8591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:20.707818 8591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:20.708591 8591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:20.708658 8591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.708725 8591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:20.708765 8591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.717255 8591 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:44081
I20260808 18:18:20.717279 8788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:44081 every 8 connection(s)
I20260808 18:18:20.718021 8591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:20.724987 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 8591
I20260808 18:18:20.725134 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:20.725464 27723 external_mini_cluster.cc:1428] Setting key 321c4a6640f5358484087f6a009081ca
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:20.730777 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.719580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:55743 (local address 127.27.18.254:34043)
0808 18:18:20.719754 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:20.719759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:20.720517 (+ 758us) server_negotiation.cc:407] Connection header received
0808 18:18:20.721659 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:20.721662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:20.721709 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:20.721779 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:20.722763 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.723255 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.724366 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.724614 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.724693 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.727492 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:20.727495 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:20.727497 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:20.727519 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:20.728756 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.729330 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.729333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.729335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.729411 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.729741 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.729744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.729746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.729925 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:20.730037 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:20.730571 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 18:18:20.730666 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 18:18:20.731845 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.719943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:55743)
0808 18:18:20.720380 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:20.720417 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:20.721462 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:20.721929 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:20.721935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:20.722233 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:20.722634 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:20.722646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.723358 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.723360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:20.724249 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:20.724258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.724760 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.724765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:20.724970 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.725610 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:20.725631 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:20.727280 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:20.728881 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:20.728890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:20.728906 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:20.729226 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:20.729496 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:20.729499 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:20.729501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:20.729626 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:20.730038 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:20.730045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:20.730319 (+ 274us) client_negotiation.cc:769] Sending connection context
0808 18:18:20.730511 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 18:18:20.730826 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260808 18:18:20.732998 8791 heartbeater.cc:347] Connected to a master server at 127.27.18.254:34043
I20260808 18:18:20.733246 8791 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:20.733836 8791 heartbeater.cc:511] Master 127.27.18.254:34043 requested a full tablet report, sending...
I20260808 18:18:20.734910 8261 ts_manager.cc:194] Registered new tserver with Master: 2edf68af071c48fdb22278eb02120faf (127.27.18.194:44081)
I20260808 18:18:20.735508 8261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:55743
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:18:20.752385 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34043
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:20.856498 8797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:20.856706 8797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:20.856745 8797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:20.860373 8797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:20.860422 8797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:20.860471 8797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:20.864986 8797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:20.865744 8797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:20.866595 8797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:20.875560 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:20.875612 8809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:20.875629 8797 server_base.cc:1034] running on GCE node
W20260808 18:18:20.875562 8804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:20.876143 8797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:20.876693 8797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:20.877848 8797 hybrid_clock.cc:648] HybridClock initialized: now 1786213100877822 us; error 37 us; skew 500 ppm
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:20.880390 8797 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:18:20.881211 8797 webserver.cc:460] Webserver started at http://127.27.18.195:40941/ using document root <none> and password file <none>
I20260808 18:18:20.881687 8797 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:20.881768 8797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:20.881970 8797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:20.883749 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "b475c9abc0484790a2282317efc6c6fe"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "85e71c09762122c211d5ccdbe7047d5b"
server_key_iv: "2514497fcc411de00e3669f3aca3129d"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.884250 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "b475c9abc0484790a2282317efc6c6fe"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "85e71c09762122c211d5ccdbe7047d5b"
server_key_iv: "2514497fcc411de00e3669f3aca3129d"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.887603 8797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:20.889622 8821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.890614 8797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:20.890734 8797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "b475c9abc0484790a2282317efc6c6fe"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "85e71c09762122c211d5ccdbe7047d5b"
server_key_iv: "2514497fcc411de00e3669f3aca3129d"
server_key_version: "encryptionkey@0"
I20260808 18:18:20.890834 8797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:20.905306 8797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:20.907718 8797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:20.907920 8797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:20.908416 8797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:20.909173 8797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:20.909237 8797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.909298 8797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:20.909348 8797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:20.917917 8797 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42423
I20260808 18:18:20.917930 8990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42423 every 8 connection(s)
I20260808 18:18:20.918725 8797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
Aug 08 18:18:20 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213100, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:20.927937 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 8797
I20260808 18:18:20.928062 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:20.928340 27723 external_mini_cluster.cc:1428] Setting key afcd36235c0b08e83bffe6f1cd2e5771
I20260808 18:18:20.931823 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.920412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:33439 (local address 127.27.18.254:34043)
0808 18:18:20.920558 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:20.920561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:20.921237 (+ 676us) server_negotiation.cc:407] Connection header received
0808 18:18:20.922468 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:20.922470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:20.922519 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:20.922591 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:20.923686 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.924204 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.925615 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.925902 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.926010 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.928449 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:20.928452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:20.928455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:20.928484 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:20.930021 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.930536 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.930539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.930540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.930607 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:20.930893 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:20.930894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:20.930895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:20.931046 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:20.931116 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:20.931613 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 18:18:20.931705 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:18:20.932569 8995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:20.920694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:33439)
0808 18:18:20.921089 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:20.921125 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:20.922285 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:20.922738 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:20.922744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:20.923061 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:20.923573 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:20.923587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.924304 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.924307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:20.925503 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:20.925514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:20.926046 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:20.926050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:20.926219 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:20.926789 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:20.926810 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:20.928287 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:20.930123 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:20.930128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:20.930140 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:20.930458 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:20.930677 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:20.930679 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:20.930680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:20.930819 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:20.931125 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:20.931129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:20.931317 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 18:18:20.931562 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 18:18:20.931782 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260808 18:18:20.933542 8992 heartbeater.cc:347] Connected to a master server at 127.27.18.254:34043
I20260808 18:18:20.933722 8992 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:20.934113 8992 heartbeater.cc:511] Master 127.27.18.254:34043 requested a full tablet report, sending...
I20260808 18:18:20.934963 8261 ts_manager.cc:194] Registered new tserver with Master: b475c9abc0484790a2282317efc6c6fe (127.27.18.195:42423)
I20260808 18:18:20.935555 8261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:33439
I20260808 18:18:20.942080 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab.
Entry for principal oryx/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab.
WARNING: no policy specified for oryx/127.27.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab.
Entry for principal oryx/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab.
WARNING: no policy specified for oryx/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab.
Entry for principal oryx/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab.
I20260808 18:18:21.005749 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 8375
I20260808 18:18:21.016418 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 8591
I20260808 18:18:21.039108 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 8797
I20260808 18:18:21.046198 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 8212
I20260808 18:18:21.056614 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:34043
--webserver_interface=127.27.18.254
--webserver_port=42041
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:34043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:21.160003 9002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:21.160226 9002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:21.160297 9002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:21.164304 9002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:21.164364 9002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:21.164430 9002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:21.164481 9002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:21.164515 9002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:21.169207 9002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:34043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:34043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=42041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:21.170027 9002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:21.170849 9002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:21.178614 9014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:21.178637 9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:21.178619 9002 server_base.cc:1034] running on GCE node
W20260808 18:18:21.178640 9017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:21.179291 9002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:21.180315 9002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:21.181483 9002 hybrid_clock.cc:648] HybridClock initialized: now 1786213101181447 us; error 29 us; skew 500 ppm
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213101, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:21.183969 9002 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:18:21.184752 9002 webserver.cc:460] Webserver started at http://127.27.18.254:42041/ using document root <none> and password file <none>
I20260808 18:18:21.185240 9002 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:21.185339 9002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:21.188760 9002 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:21.190632 9028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.191699 9002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:21.191820 9002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "50bb831405d645d4840013d99911b345"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "45ac3ede496d410a3350e8b6b3018b63"
server_key_iv: "8662324fb9991c0558a91c53410a830f"
server_key_version: "encryptionkey@0"
I20260808 18:18:21.192260 9002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:21.219424 9002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:21.221844 9002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:21.222018 9002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:21.228634 9002 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:34043
I20260808 18:18:21.228644 9113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:34043 every 8 connection(s)
I20260808 18:18:21.229507 9002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:21.232302 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 9002
I20260808 18:18:21.232806 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:32909
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:34043
--webserver_port=44735
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:18:21.233771 9114 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:18:21.235695 9114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Bootstrap starting.
I20260808 18:18:21.251925 9114 log.cc:824] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:21.257619 9114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:18:21.257930 9114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Bootstrap complete.
I20260808 18:18:21.261651 9114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:21.262008 9114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bb831405d645d4840013d99911b345, State: Initialized, Role: FOLLOWER
I20260808 18:18:21.262446 9114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:21.262555 9114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:21.262635 9114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:21.262746 9114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:18:21.264571 9114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:21.264914 9114 leader_election.cc:302] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50bb831405d645d4840013d99911b345; no voters:
I20260808 18:18:21.265208 9114 leader_election.cc:288] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:18:21.265326 9121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:18:21.265554 9121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [term 2 LEADER]: Becoming Leader. State: Replica: 50bb831405d645d4840013d99911b345, State: Running, Role: LEADER
I20260808 18:18:21.265846 9121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } }
I20260808 18:18:21.266315 9114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:21.267089 9122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "50bb831405d645d4840013d99911b345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } } }
I20260808 18:18:21.267210 9122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:21.267923 9124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50bb831405d645d4840013d99911b345. Latest consensus state: current_term: 2 leader_uuid: "50bb831405d645d4840013d99911b345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50bb831405d645d4840013d99911b345" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34043 } } }
I20260808 18:18:21.268019 9124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:21.268326 9134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:21.271498 9134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:21.272053 9134 catalog_manager.cc:1259] Loaded cluster ID: 0b3d4b78434b43c0b1e5016b8f24eb53
I20260808 18:18:21.272110 9134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:21.274356 9134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:21.274994 9134 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50bb831405d645d4840013d99911b345: Loaded TSK: 0
I20260808 18:18:21.276407 9134 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:18:21.341239 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:21.341444 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:21.341480 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:21.345175 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:21.345227 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:21.345306 9117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:21.350593 9117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.193.keytab
--principal=oryx/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:32909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=44735
--tserver_master_addrs=127.27.18.254:34043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:21.351540 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:21.352437 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:21.361622 9151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:21.361636 9153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:21.361893 9117 server_base.cc:1034] running on GCE node
W20260808 18:18:21.362058 9156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:21.362521 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:21.363194 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:21.364403 9117 hybrid_clock.cc:648] HybridClock initialized: now 1786213101364372 us; error 35 us; skew 500 ppm
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213101, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:21.367810 9117 init.cc:377] Logged in from keytab as oryx/127.27.18.193@KRBTEST.COM (short username oryx)
I20260808 18:18:21.368901 9117 webserver.cc:460] Webserver started at http://127.27.18.193:44735/ using document root <none> and password file <none>
I20260808 18:18:21.369587 9117 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:21.369699 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:21.374372 9117 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 18:18:21.376775 9166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.378188 9117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:18:21.378350 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "b39877d6e8e441928d26ac89eaf534b6"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "c364cee8d66acc07e4e63cccdb389d8d"
server_key_iv: "7ee26dd817e84fa920d2da9f02610986"
server_key_version: "encryptionkey@0"
I20260808 18:18:21.378974 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:21.408339 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:21.411474 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:21.411610 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:21.412067 9117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:21.412766 9117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:21.412808 9117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.412842 9117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:21.412859 9117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.421159 9117 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:32909
I20260808 18:18:21.421244 9336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:32909 every 8 connection(s)
I20260808 18:18:21.422061 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:21.429631 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 9117
I20260808 18:18:21.430047 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:44081
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:34043
--webserver_port=33129
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:18:21.433882 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.423995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:46795 (local address 127.27.18.254:34043)
0808 18:18:21.424388 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.424403 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.424615 (+ 212us) server_negotiation.cc:407] Connection header received
0808 18:18:21.426008 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.426017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.426284 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.426539 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.427579 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.428476 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.429549 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.430025 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.430191 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.433264 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:46795: BlockingRecv error: recv got EOF from 127.27.18.193:46795 (error 108)
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":121,"threads_started":1}
I20260808 18:18:21.433892 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.424055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:46795)
0808 18:18:21.424470 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.424515 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.425684 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.426649 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.426655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.426971 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.427429 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.427446 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.428596 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.428599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.429425 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.429434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.430191 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.430194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.430364 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.430842 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.430857 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.433070 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":123,"threads_started":1}
W20260808 18:18:21.434504 9337 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:34043 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.18.254:34043: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:21.439566 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.434869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:56287)
0808 18:18:21.434966 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.434980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.435045 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.435280 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.435300 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.435554 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.435639 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.435644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.436335 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.436338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.436860 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.436867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.437280 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.437283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.437381 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.437751 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.437767 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.439446 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:18:21.439625 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.434975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:56287 (local address 127.27.18.254:34043)
0808 18:18:21.435091 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.435095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.435107 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:21.435149 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.435151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.435182 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.435261 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.435747 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.436222 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.436974 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.437181 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.437278 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.439532 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:56287: BlockingRecv error: recv got EOF from 127.27.18.193:56287 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:21.444500 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.440202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:36507)
0808 18:18:21.440299 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.440312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.440393 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.440612 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.440614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.440755 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.440821 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.440826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.441444 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.441448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.441897 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.441904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.442289 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.442292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.442364 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.442730 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.442740 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.444361 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:18:21.444587 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.440317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36507 (local address 127.27.18.254:34043)
0808 18:18:21.440439 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.440443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.440455 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:21.440489 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.440490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.440529 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.440596 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.440930 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.441348 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.442009 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.442193 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.442286 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.444481 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:36507: BlockingRecv error: recv got EOF from 127.27.18.193:36507 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 18:18:21.444813 9337 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:18:21.537214 9342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:21.537396 9342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:21.537432 9342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:21.541193 9342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:21.541234 9342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:21.541283 9342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:21.545977 9342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.194.keytab
--principal=oryx/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:44081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=33129
--tserver_master_addrs=127.27.18.254:34043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:21.546756 9342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:21.547596 9342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:18:21.555842 9342 server_base.cc:1034] running on GCE node
W20260808 18:18:21.555914 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:21.555899 9352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:21.556083 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:21.556542 9342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:21.557071 9342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:21.558240 9342 hybrid_clock.cc:648] HybridClock initialized: now 1786213101558212 us; error 38 us; skew 500 ppm
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213101, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:21.560721 9342 init.cc:377] Logged in from keytab as oryx/127.27.18.194@KRBTEST.COM (short username oryx)
I20260808 18:18:21.561532 9342 webserver.cc:460] Webserver started at http://127.27.18.194:33129/ using document root <none> and password file <none>
I20260808 18:18:21.562026 9342 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:21.562144 9342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:21.565569 9342 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260808 18:18:21.567696 9366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.568782 9342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 18:18:21.568905 9342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "2edf68af071c48fdb22278eb02120faf"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "1836604c6adf1faeae2255402abaabe0"
server_key_iv: "0c889b4ac4c3913b6480b6062e36979b"
server_key_version: "encryptionkey@0"
I20260808 18:18:21.569403 9342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:21.584889 9342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:21.587663 9342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:21.587875 9342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:21.588390 9342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:21.589129 9342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:21.589210 9342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.589269 9342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:21.589320 9342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.597971 9342 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:44081
I20260808 18:18:21.598014 9565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:44081 every 8 connection(s)
I20260808 18:18:21.598913 9342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:21.606864 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 9342
I20260808 18:18:21.607318 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:42423
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:34043
--webserver_port=40941
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:18:21.609320 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.600481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:59061 (local address 127.27.18.254:34043)
0808 18:18:21.600606 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.600609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.601372 (+ 763us) server_negotiation.cc:407] Connection header received
0808 18:18:21.602683 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.602686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.602751 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.602861 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.603966 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.604503 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.605623 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.605852 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.605960 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.609200 (+ 3240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:59061: BlockingRecv error: recv got EOF from 127.27.18.194:59061 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:18:21.609836 9570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.600791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:59061)
0808 18:18:21.601227 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.601268 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.602486 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.603019 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.603026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.603355 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.603832 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.603845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.604637 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.604640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.605491 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.605501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.605997 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.606002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.606159 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.606686 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.606701 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.609087 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":111,"threads_started":1}
W20260808 18:18:21.610317 9568 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:34043 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.18.254:34043: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:21.615922 9570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.610716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:49293)
0808 18:18:21.610821 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.610833 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.610899 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.611130 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.611131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.611239 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.611320 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.611324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.612360 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.612362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.612863 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.612871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.613498 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.613501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.613565 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.613996 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.614008 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.615841 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 18:18:21.616174 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.610795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:49293 (local address 127.27.18.254:34043)
0808 18:18:21.610920 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.610926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.610941 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:18:21.610981 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.610986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.611032 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.611116 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.611453 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.612245 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.612981 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.613391 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.613516 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.616013 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:49293: BlockingRecv error: recv got EOF from 127.27.18.194:49293 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:21.621477 9570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.616543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:42269)
0808 18:18:21.616700 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.616712 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.616774 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.617023 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.617025 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.617139 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.617201 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.617205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.618220 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.618222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.618686 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.618692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.619278 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.619280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.619337 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.619779 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.619790 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.621401 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260808 18:18:21.621642 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.616646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:42269 (local address 127.27.18.254:34043)
0808 18:18:21.616783 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.616790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.616806 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:18:21.616863 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.616869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.616920 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.617012 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.617315 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.618116 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.618790 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.619169 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.619292 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.621539 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:42269: BlockingRecv error: recv got EOF from 127.27.18.194:42269 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 18:18:21.621706 9568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:18:21.716008 9572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:21.716225 9572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:21.716264 9572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:21.720053 9572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:21.720096 9572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:21.720177 9572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:21.725062 9572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/krb5kdc/oryx_127.27.18.195.keytab
--principal=oryx/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:42423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=40941
--tserver_master_addrs=127.27.18.254:34043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:21.725952 9572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:21.726832 9572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:21.735742 9579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:21.735776 9583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:21.735883 9580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:21.736102 9572 server_base.cc:1034] running on GCE node
I20260808 18:18:21.736462 9572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:21.736994 9572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:21.738180 9572 hybrid_clock.cc:648] HybridClock initialized: now 1786213101738162 us; error 39 us; skew 500 ppm
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213101, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:21.740759 9572 init.cc:377] Logged in from keytab as oryx/127.27.18.195@KRBTEST.COM (short username oryx)
I20260808 18:18:21.741534 9572 webserver.cc:460] Webserver started at http://127.27.18.195:40941/ using document root <none> and password file <none>
I20260808 18:18:21.742007 9572 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:21.742072 9572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:21.745553 9572 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:21.747434 9593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.748550 9572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:21.748653 9572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "b475c9abc0484790a2282317efc6c6fe"
format_stamp: "Formatted at 2026-08-08 18:18:20 on dist-test-slave-x4qh"
server_key: "85e71c09762122c211d5ccdbe7047d5b"
server_key_iv: "2514497fcc411de00e3669f3aca3129d"
server_key_version: "encryptionkey@0"
I20260808 18:18:21.749150 9572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:21.763778 9572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:21.766371 9572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:21.766508 9572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:21.767009 9572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:21.767966 9572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:21.768013 9572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.768070 9572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:21.768101 9572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:21.777496 9572 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42423
I20260808 18:18:21.777529 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42423 every 8 connection(s)
I20260808 18:18:21.778429 9572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:21.784847 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 9572
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 18:18:21.789505 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.780080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:34107 (local address 127.27.18.254:34043)
0808 18:18:21.780217 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.780221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.781011 (+ 790us) server_negotiation.cc:407] Connection header received
0808 18:18:21.782406 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.782409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.782466 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.782538 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.783689 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.784208 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.785520 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.785845 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.785980 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.789391 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:34107: BlockingRecv error: recv got EOF from 127.27.18.195:34107 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:18:21.790145 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.780462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:34107)
0808 18:18:21.780879 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.780918 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.782176 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.782693 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.782699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.783014 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.783535 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.783551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.784337 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.784339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.785403 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.785414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.786033 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.786037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.786215 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.786790 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.786808 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.789289 (+ 2481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
W20260808 18:18:21.790707 9757 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:34043 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.18.254:34043: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
I20260808 18:18:21.794852 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.785839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.27.18.254:34043)
0808 18:18:21.786106 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.786110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.786120 (+ 10us) server_negotiation.cc:407] Connection header received
0808 18:18:21.786150 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.786151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.786197 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.786320 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.787061 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.787770 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.788887 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.789091 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.789190 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.790733 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:21.790739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:21.790766 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:21.790799 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:21.792689 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:21.793107 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:21.793112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:21.793118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:21.793252 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:21.793532 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:21.793534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:21.793535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:21.793938 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:21.794107 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:21.794425 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 18:18:21.794647 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":104,"threads_started":1}
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:21.796226 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.791092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:51203)
0808 18:18:21.791282 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.791294 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.791360 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.791623 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.791626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.791789 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.791932 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.791940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.792607 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.792612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.793289 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.793300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.793744 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.793764 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.793856 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.794301 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.794316 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.796106 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124}
W20260808 18:18:21.796347 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.791181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:51203 (local address 127.27.18.254:34043)
0808 18:18:21.791328 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.791334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.791349 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:18:21.791487 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.791489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.791522 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.791581 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.792053 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.792498 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.793396 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.793643 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.793711 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.796218 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:51203: BlockingRecv error: recv got EOF from 127.27.18.195:51203 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:21 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:21.801009 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:21.796782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:60611)
0808 18:18:21.796900 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:21.796913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:21.796982 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:21.797332 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:21.797335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:21.797481 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:21.797566 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.797572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.798217 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.798219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.798670 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:21.798677 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.799063 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.799065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:21.799117 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.799496 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:21.799507 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:21.800939 (+ 1432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 18:18:21.801118 9339 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:21.797028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:60611 (local address 127.27.18.254:34043)
0808 18:18:21.797159 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:21.797163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:21.797174 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:18:21.797208 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:21.797210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:21.797238 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:21.797292 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:21.797664 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.798133 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.798783 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:21.798976 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:21.799048 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:21.801041 (+ 1993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:60611: BlockingRecv error: recv got EOF from 127.27.18.195:60611 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 18:18:21.801467 9757 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 18:18:22 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:22 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:22.450770 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:22.445458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:59109)
0808 18:18:22.445731 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:22.445743 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:22.445830 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:22.446098 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:22.446101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:22.446268 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:22.446374 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:22.446379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.447283 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.447287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:22.448019 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:22.448028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.448429 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.448432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:22.448491 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:22.448942 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:22.448953 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:22.450656 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
W20260808 18:18:22.450951 9763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:22.445540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:59109 (local address 127.27.18.254:34043)
0808 18:18:22.445778 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:22.445781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:22.445794 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:22.445920 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:22.445922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:22.445984 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:22.446061 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:22.446474 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.447145 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.448117 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.448319 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.448414 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:22.450776 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:59109: BlockingRecv error: recv got EOF from 127.27.18.193:59109 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
Aug 08 18:18:22 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:22 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:22.627487 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:22.622374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:33121)
0808 18:18:22.622626 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:22.622639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:22.622732 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:22.622957 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:22.622959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:22.623133 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:22.623247 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:22.623252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.624020 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.624022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:22.624591 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:22.624599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.624982 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.624984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:22.625054 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:22.625601 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:22.625613 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:22.627338 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
W20260808 18:18:22.627630 9763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:22.622444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:33121 (local address 127.27.18.254:34043)
0808 18:18:22.622593 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:22.622599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:22.622709 (+ 110us) server_negotiation.cc:407] Connection header received
0808 18:18:22.622810 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:22.622812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:22.622856 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:22.622924 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:22.623347 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.623927 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.624689 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.624890 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.624962 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:22.627517 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:33121: BlockingRecv error: recv got EOF from 127.27.18.194:33121 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 18:18:22 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:22 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:22.807279 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:22.802247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:56147)
0808 18:18:22.802520 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:22.802536 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:22.802670 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:22.802879 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:22.802881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:22.803039 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:22.803130 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:22.803135 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.803836 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.803838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:22.804429 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:22.804438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.804844 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.804847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:22.804908 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:22.805419 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:22.805430 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:22.807168 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
W20260808 18:18:22.807453 9763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:22.802339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:56147 (local address 127.27.18.254:34043)
0808 18:18:22.802509 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:22.802514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:22.802634 (+ 120us) server_negotiation.cc:407] Connection header received
0808 18:18:22.802735 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:22.802737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:22.802780 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:22.802844 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:22.803225 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.803742 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.804537 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:22.804747 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:22.804821 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:22.807292 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:56147: BlockingRecv error: recv got EOF from 127.27.18.195:56147 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 18:18:23 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:23 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:23.456710 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:23.451541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:54663)
0808 18:18:23.451809 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:23.451826 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:23.451918 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:23.452163 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:23.452165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:23.452345 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:23.452443 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:23.452448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.453233 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.453235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:23.453773 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:23.453780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.454202 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.454205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:23.454269 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:23.454741 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:23.454751 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:23.456547 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
W20260808 18:18:23.456802 9771 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:23.451578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:54663 (local address 127.27.18.254:34043)
0808 18:18:23.451862 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:23.451866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:23.451880 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:23.452002 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:23.452004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:23.452048 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:23.452124 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:23.452539 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.453135 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.453886 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.454104 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.454179 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:23.456664 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:54663: BlockingRecv error: recv got EOF from 127.27.18.193:54663 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
Aug 08 18:18:23 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:23 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:23.633646 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:23.628239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:54799)
0808 18:18:23.628495 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:23.628509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:23.628627 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:23.628842 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:23.628844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:23.629013 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:23.629111 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:23.629116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.629823 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.629838 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:23.630392 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:23.630400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.630905 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.630909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:23.630998 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:23.631607 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:23.631626 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:23.633463 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
W20260808 18:18:23.633670 9771 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:23.628286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:54799 (local address 127.27.18.254:34043)
0808 18:18:23.628431 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:23.628434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:23.628603 (+ 169us) server_negotiation.cc:407] Connection header received
0808 18:18:23.628714 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:23.628717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:23.628761 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:23.628832 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:23.629220 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.629728 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.630515 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.630772 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.630878 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:23.633576 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:54799: BlockingRecv error: recv got EOF from 127.27.18.194:54799 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 18:18:23 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:23 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:23.816767 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:23.808154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:34517)
0808 18:18:23.808445 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:23.808457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:23.808544 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:23.808775 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:23.808777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:23.808933 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:23.809024 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:23.809029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.809666 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.809668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:23.810158 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:23.810164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.810531 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.810534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:23.810608 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:23.814773 (+ 4165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:23.814785 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:23.816605 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":113,"threads_started":1}
W20260808 18:18:23.816831 9771 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:23.808276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:34517 (local address 127.27.18.254:34043)
0808 18:18:23.808413 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:23.808416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:23.808521 (+ 105us) server_negotiation.cc:407] Connection header received
0808 18:18:23.808617 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:23.808619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:23.808662 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:23.808740 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:23.809115 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.809579 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.810247 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:23.810437 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:23.810510 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:23.816728 (+ 6218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:34517: BlockingRecv error: recv got EOF from 127.27.18.195:34517 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 18:18:24 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:24 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:24.463032 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:24.457705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:38805)
0808 18:18:24.458017 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:24.458030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:24.458125 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:24.458365 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:24.458368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:24.458538 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:24.458648 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:24.458653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.459319 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.459321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:24.459870 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:24.459878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.460264 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.460266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:24.460332 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:24.460769 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:24.460780 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:24.462901 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":124,"threads_started":1}
W20260808 18:18:24.463157 9778 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:24.457766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:38805 (local address 127.27.18.254:34043)
0808 18:18:24.458034 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:24.458037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:24.458099 (+ 62us) server_negotiation.cc:407] Connection header received
0808 18:18:24.458203 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:24.458206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:24.458248 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:24.458327 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:24.458741 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.459225 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.459963 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.460171 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.460243 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:24.463027 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:38805: BlockingRecv error: recv got EOF from 127.27.18.193:38805 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":119,"threads_started":1}
Aug 08 18:18:24 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:24 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:24.640022 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:24.634419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:49367)
0808 18:18:24.634718 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:24.634733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:24.634879 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:24.635144 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:24.635147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:24.635370 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:24.635524 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:24.635530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.636268 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.636273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:24.636853 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:24.636862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.637331 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.637335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:24.637430 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:24.638025 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:24.638036 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:24.639909 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
W20260808 18:18:24.640166 9778 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:24.634490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:49367 (local address 127.27.18.254:34043)
0808 18:18:24.634676 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:24.634698 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:24.634818 (+ 120us) server_negotiation.cc:407] Connection header received
0808 18:18:24.634977 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:24.634981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:24.635036 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:24.635119 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:24.635635 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.636136 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.636977 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.637210 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.637327 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:24.640050 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:49367: BlockingRecv error: recv got EOF from 127.27.18.194:49367 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 18:18:24 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:24 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:24.822896 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:24.817517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:40395)
0808 18:18:24.817945 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:24.817960 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:24.818089 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:24.818346 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:24.818348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:24.818530 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:24.818618 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:24.818623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.819348 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.819352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:24.819921 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:24.819929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.820399 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.820402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:24.820481 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:24.821022 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:24.821033 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:24.822763 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":93,"threads_started":1}
W20260808 18:18:24.823033 9778 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:24.817698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:40395 (local address 127.27.18.254:34043)
0808 18:18:24.817945 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:24.817950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:24.818035 (+ 85us) server_negotiation.cc:407] Connection header received
0808 18:18:24.818178 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:24.818181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:24.818233 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:24.818310 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:24.818728 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.819235 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.820067 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:24.820294 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:24.820395 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:24.822894 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:40395: BlockingRecv error: recv got EOF from 127.27.18.195:40395 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 08 18:18:25 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:25 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:25.469771 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:25.463896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:41299)
0808 18:18:25.464337 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:25.464360 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:25.464487 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:25.464900 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:25.464903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:25.465117 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:25.465217 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:25.465223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.465964 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.465967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:25.466608 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:25.466616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.467089 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.467092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:25.467154 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:25.467683 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:25.467695 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:25.469610 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":100,"threads_started":1}
W20260808 18:18:25.470006 9784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:25.464010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:41299 (local address 127.27.18.254:34043)
0808 18:18:25.464270 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:25.464275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:25.464451 (+ 176us) server_negotiation.cc:407] Connection header received
0808 18:18:25.464597 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:25.464600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:25.464658 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:25.464745 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:25.465317 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.465843 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.466704 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.466950 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.467025 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:25.469726 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:41299: BlockingRecv error: recv got EOF from 127.27.18.193:41299 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":59,"threads_started":1}
Aug 08 18:18:25 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:25 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:25.646432 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:25.640849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:59007)
0808 18:18:25.641119 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:25.641136 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:25.641239 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:25.641541 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:25.641543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:25.641746 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:25.641853 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:25.641859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.642611 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.642616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:25.643163 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:25.643171 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.643671 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.643674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:25.643761 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:25.644231 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:25.644243 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:25.646264 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
W20260808 18:18:25.646489 9784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:25.641073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:59007 (local address 127.27.18.254:34043)
0808 18:18:25.641212 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:25.641217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:25.641231 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:25.641361 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:25.641376 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:25.641419 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:25.641505 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:25.641979 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.642498 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.643286 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.643560 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.643666 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:25.646382 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:59007: BlockingRecv error: recv got EOF from 127.27.18.194:59007 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 18:18:25 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:25 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:25.829308 9787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:25.823754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:34515)
0808 18:18:25.824039 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:25.824052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:25.824189 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:25.824473 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:25.824476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:25.824682 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:25.824778 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:25.824783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.825529 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.825533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:25.826054 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:25.826061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.826523 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.826527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:25.826621 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:25.827078 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:25.827090 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:25.829106 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
W20260808 18:18:25.829731 9784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:25.823814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:34515 (local address 127.27.18.254:34043)
0808 18:18:25.823967 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:25.823972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:25.824130 (+ 158us) server_negotiation.cc:407] Connection header received
0808 18:18:25.824291 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:25.824297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:25.824363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:25.824451 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:25.824910 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.825420 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.826166 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:25.826406 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:25.826509 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:25.829598 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:34515: BlockingRecv error: recv got EOF from 127.27.18.195:34515 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 18:18:26 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:26 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:26.476226 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:26.470489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:38373)
0808 18:18:26.470735 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:26.470747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:26.470844 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:26.471067 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:26.471069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:26.471238 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:26.471331 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:26.471336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.472384 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.472403 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:26.473142 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:26.473153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.473595 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.473597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:26.473658 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:26.474212 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:26.474226 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:26.476066 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
W20260808 18:18:26.476329 9790 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:26.470534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:38373 (local address 127.27.18.254:34043)
0808 18:18:26.470752 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:26.470756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:26.470818 (+ 62us) server_negotiation.cc:407] Connection header received
0808 18:18:26.470921 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:26.470923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:26.470966 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:26.471038 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:26.471486 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.472242 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.473273 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.473500 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.473578 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:26.476181 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:38373: BlockingRecv error: recv got EOF from 127.27.18.193:38373 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":61,"threads_started":1}
Aug 08 18:18:26 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:26 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:26.652370 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:26.647276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:56047)
0808 18:18:26.647566 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:26.647579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:26.647661 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:26.647898 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:26.647900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:26.648069 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:26.648160 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:26.648165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.648817 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.648820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:26.649345 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:26.649352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.649750 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.649753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:26.649822 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:26.650250 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:26.650277 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:26.652185 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
W20260808 18:18:26.652410 9790 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:26.647448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:56047 (local address 127.27.18.254:34043)
0808 18:18:26.647610 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:26.647614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:26.647627 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:26.647754 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:26.647756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:26.647799 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:26.647863 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:26.648247 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.648725 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.649454 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.649659 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.649729 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:26.652301 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:56047: BlockingRecv error: recv got EOF from 127.27.18.194:56047 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 18:18:26 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:26 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:26.836175 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:26.830263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:40661)
0808 18:18:26.830600 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:26.830619 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:26.830748 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:26.831036 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:26.831040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:26.831244 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:26.831349 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:26.831354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.832140 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.832144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:26.832735 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:26.832743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.833209 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.833213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:26.833298 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:26.834010 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:26.834024 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:26.836064 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":89,"threads_started":1}
W20260808 18:18:26.836341 9790 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:26.830402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:40661 (local address 127.27.18.254:34043)
0808 18:18:26.830570 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:26.830575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:26.830695 (+ 120us) server_negotiation.cc:407] Connection header received
0808 18:18:26.830864 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:26.830867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:26.830923 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:26.831010 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:26.831506 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.832024 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.832871 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:26.833097 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:26.833204 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:26.836186 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:40661: BlockingRecv error: recv got EOF from 127.27.18.195:40661 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 18:18:27 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:27 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:27.482055 9795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:27.477074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:34005)
0808 18:18:27.477376 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:27.477390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:27.477480 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:27.477736 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:27.477738 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:27.477912 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:27.478010 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:27.478015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.478690 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.478692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:27.479244 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:27.479251 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.479662 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.479665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:27.479729 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:27.480270 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:27.480282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:27.481951 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
W20260808 18:18:27.482223 9796 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:27.477135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:34005 (local address 127.27.18.254:34043)
0808 18:18:27.477422 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:27.477426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:27.477439 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:18:27.477576 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:27.477579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:27.477623 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:27.477699 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:27.478106 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.478604 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.479343 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.479572 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.479644 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:27.482068 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:34005: BlockingRecv error: recv got EOF from 127.27.18.193:34005 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":122,"threads_started":1}
Aug 08 18:18:27 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:27 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:27.658694 9797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:27.653172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:34637)
0808 18:18:27.653444 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:27.653456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:27.653547 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:27.653838 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:27.653840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:27.654077 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:27.654217 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:27.654227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.655177 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.655179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:27.655743 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:27.655752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.656149 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.656152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:27.656210 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:27.656703 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:27.656715 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:27.658545 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":91,"threads_started":1}
W20260808 18:18:27.658936 9796 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:27.653251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:34637 (local address 127.27.18.254:34043)
0808 18:18:27.653408 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:27.653412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:27.653520 (+ 108us) server_negotiation.cc:407] Connection header received
0808 18:18:27.653660 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:27.653662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:27.653725 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:27.653812 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:27.654348 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.655083 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.655837 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.656041 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.656131 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:27.658699 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:34637: BlockingRecv error: recv got EOF from 127.27.18.194:34637 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 18:18:27 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:27 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:27.842015 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:27.837043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:32841)
0808 18:18:27.837305 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:27.837346 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:27.837472 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:27.837702 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:27.837704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:27.837885 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:27.837976 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:27.837981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.838632 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.838634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:27.839138 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:27.839145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.839558 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.839561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:27.839621 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:27.840090 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:27.840102 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:27.841904 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
W20260808 18:18:27.842162 9796 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:27.837099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:32841 (local address 127.27.18.254:34043)
0808 18:18:27.837251 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:27.837256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:27.837452 (+ 196us) server_negotiation.cc:407] Connection header received
0808 18:18:27.837580 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:27.837583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:27.837629 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:27.837691 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:27.838068 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.838544 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.839229 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:27.839478 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:27.839549 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:27.842036 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:32841: BlockingRecv error: recv got EOF from 127.27.18.195:32841 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 18:18:28 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:28 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:28.488015 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:28.482733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:36879)
0808 18:18:28.483067 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:28.483078 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:28.483167 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:28.483481 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:28.483483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:28.483648 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:28.483761 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:28.483766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.484527 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.484530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:28.485063 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:28.485070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.485500 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.485515 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:28.485571 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:28.485999 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:28.486010 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:28.487898 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":79,"threads_started":1}
W20260808 18:18:28.488162 9805 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:28.482785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36879 (local address 127.27.18.254:34043)
0808 18:18:28.483011 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:28.483015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:28.483121 (+ 106us) server_negotiation.cc:407] Connection header received
0808 18:18:28.483257 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:28.483259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:28.483330 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:28.483442 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:28.483864 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.484419 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.485184 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.485405 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.485478 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:28.488042 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:36879: BlockingRecv error: recv got EOF from 127.27.18.193:36879 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
Aug 08 18:18:28 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:28 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:28.665473 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:28.659732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:49943)
0808 18:18:28.660138 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:28.660156 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:28.660290 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:28.660618 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:28.660622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:28.660926 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:28.661032 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:28.661038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.661737 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.661739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:28.662299 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:28.662307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.662716 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.662719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:28.662782 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:28.663237 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:28.663248 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:28.665278 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":98,"threads_started":1}
W20260808 18:18:28.665508 9805 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:28.659784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:49943 (local address 127.27.18.254:34043)
0808 18:18:28.659979 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:28.659983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:28.660224 (+ 241us) server_negotiation.cc:407] Connection header received
0808 18:18:28.660386 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:28.660390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:28.660461 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:28.660567 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:28.661138 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.661638 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.662403 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.662623 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.662695 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:28.665411 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:49943: BlockingRecv error: recv got EOF from 127.27.18.194:49943 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 08 18:18:28 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:28 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:28.853036 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:28.842959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:33471)
0808 18:18:28.843324 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:28.843337 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:28.843455 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:28.843679 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:28.843681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:28.843878 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:28.843972 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:28.843976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.844684 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.844686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:28.845208 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:28.845215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.845634 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.845636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:28.845694 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:28.850862 (+ 5168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:28.850874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:28.852922 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
W20260808 18:18:28.853214 9805 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:28.843021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:33471 (local address 127.27.18.254:34043)
0808 18:18:28.843161 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:28.843165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:28.843409 (+ 244us) server_negotiation.cc:407] Connection header received
0808 18:18:28.843528 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:28.843530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:28.843575 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:28.843639 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:28.844070 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.844590 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.845305 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:28.845537 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:28.845609 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:28.853051 (+ 7442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:33471: BlockingRecv error: recv got EOF from 127.27.18.195:33471 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 18:18:29 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:29 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:29.493968 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:29.488819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:41869)
0808 18:18:29.489108 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:29.489122 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:29.489211 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:29.489434 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:29.489436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:29.489613 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:29.489721 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:29.489726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.490430 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.490433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:29.490972 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:29.490979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.491431 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.491433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:29.491495 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:29.491987 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:29.491997 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:29.493798 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":106,"threads_started":1}
W20260808 18:18:29.494052 9809 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:29.488981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:41869 (local address 127.27.18.254:34043)
0808 18:18:29.489215 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:29.489219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:29.489231 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:29.489271 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:29.489273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:29.489326 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:29.489399 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:29.489821 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.490342 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.491069 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.491310 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.491446 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:29.493925 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:41869: BlockingRecv error: recv got EOF from 127.27.18.193:41869 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
Aug 08 18:18:29 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:29 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:29.672063 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:29.666340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:41105)
0808 18:18:29.666665 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:29.666679 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:29.666822 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:29.667133 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:29.667135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:29.667321 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:29.667494 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:29.667515 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.668361 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.668365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:29.668895 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:29.668903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.669288 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.669291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:29.669352 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:29.669936 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:29.669947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:29.671921 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":115,"threads_started":1}
W20260808 18:18:29.674899 9809 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:29.666399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:41105 (local address 127.27.18.254:34043)
0808 18:18:29.666563 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:29.666568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:29.666767 (+ 199us) server_negotiation.cc:407] Connection header received
0808 18:18:29.666964 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:29.666968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:29.667014 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:29.667116 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:29.667600 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.668218 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.668996 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.669191 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.669260 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:29.674788 (+ 5528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:41105: BlockingRecv error: recv got EOF from 127.27.18.194:41105 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 18:18:29 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:29 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:29.858812 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:29.853824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:40619)
0808 18:18:29.854098 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:29.854110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:29.854202 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:29.854429 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:29.854431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:29.854599 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:29.854692 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:29.854696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.855421 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.855423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:29.855966 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:29.855973 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.856370 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.856373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:29.856432 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:29.856905 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:29.856917 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:29.858644 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
W20260808 18:18:29.858920 9809 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:29.853866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:40619 (local address 127.27.18.254:34043)
0808 18:18:29.854002 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:29.854005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:29.854174 (+ 169us) server_negotiation.cc:407] Connection header received
0808 18:18:29.854276 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:29.854278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:29.854325 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:29.854391 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:29.854782 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.855293 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.856062 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:29.856273 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:29.856350 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:29.858793 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:40619: BlockingRecv error: recv got EOF from 127.27.18.195:40619 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 18:18:30 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:30 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:30.500406 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:30.494759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:33289)
0808 18:18:30.495090 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:30.495111 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:30.495230 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:30.495584 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:30.495587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:30.495784 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:30.495883 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:30.495888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.496738 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.496743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:30.497318 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:30.497327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.497811 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.497814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:30.497899 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:30.498364 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:30.498375 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:30.500294 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":88,"threads_started":1}
W20260808 18:18:30.500597 9814 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:30.494849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:33289 (local address 127.27.18.254:34043)
0808 18:18:30.495089 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:30.495093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:30.495184 (+ 91us) server_negotiation.cc:407] Connection header received
0808 18:18:30.495346 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:30.495349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:30.495434 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:30.495559 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:30.495996 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.496610 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.497443 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.497688 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.497798 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:30.500428 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:33289: BlockingRecv error: recv got EOF from 127.27.18.193:33289 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":61,"threads_started":1}
Aug 08 18:18:30 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:30 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:30.678447 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:30.673006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:55135)
0808 18:18:30.673274 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:30.673290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:30.673398 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:30.673603 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:30.673604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:30.673777 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:30.673885 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:30.673891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.674557 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.674559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:30.675084 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:30.675091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.675545 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.675548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:30.675611 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:30.676137 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:30.676149 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:30.678289 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
W20260808 18:18:30.678503 9814 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:30.673062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:55135 (local address 127.27.18.254:34043)
0808 18:18:30.673244 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:30.673251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:30.673369 (+ 118us) server_negotiation.cc:407] Connection header received
0808 18:18:30.673465 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:30.673467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:30.673512 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:30.673579 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:30.673981 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.674472 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.675180 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.675450 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.675525 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:30.678407 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:55135: BlockingRecv error: recv got EOF from 127.27.18.194:55135 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 18:18:30 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:30 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:30.865213 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:30.859655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:58169)
0808 18:18:30.859940 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:30.859954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:30.860085 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:30.860488 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:30.860491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:30.860697 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:30.860792 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:30.860797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.861549 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.861553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:30.862109 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:30.862117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.862599 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.862603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:30.862690 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:30.863255 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:30.863265 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:30.865103 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":86,"threads_started":1}
W20260808 18:18:30.865379 9814 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:30.859696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:58169 (local address 127.27.18.254:34043)
0808 18:18:30.859846 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:30.859853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:30.860059 (+ 206us) server_negotiation.cc:407] Connection header received
0808 18:18:30.860294 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:30.860299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:30.860360 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:30.860444 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:30.860908 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.861427 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.862226 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:30.862470 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:30.862592 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:30.865228 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:58169: BlockingRecv error: recv got EOF from 127.27.18.195:58169 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 18:18:31 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:31 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:31.507014 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:31.501384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:47385)
0808 18:18:31.501689 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:31.501711 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:31.501810 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:31.502052 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:31.502054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:31.502236 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:31.502327 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:31.502332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.503029 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.503031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:31.503630 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:31.503637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.504045 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.504048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:31.504107 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:31.504673 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:31.504694 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:31.506809 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":109,"threads_started":1}
W20260808 18:18:31.507092 9820 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:31.501430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:47385 (local address 127.27.18.254:34043)
0808 18:18:31.501695 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:31.501700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:31.501782 (+ 82us) server_negotiation.cc:407] Connection header received
0808 18:18:31.501890 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:31.501892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:31.501934 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:31.502013 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:31.502419 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.502937 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.503726 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.503949 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.504025 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:31.506946 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:47385: BlockingRecv error: recv got EOF from 127.27.18.193:47385 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
Aug 08 18:18:31 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:31 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:31.684916 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:31.679356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:58243)
0808 18:18:31.679663 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:31.679675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:31.679809 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:31.680105 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:31.680108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:31.680311 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:31.680411 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:31.680417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.681160 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.681165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:31.681723 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:31.681731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.682194 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.682198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:31.682282 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:31.682843 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:31.682854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:31.684742 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":85,"threads_started":1}
W20260808 18:18:31.684953 9820 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:31.679431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:58243 (local address 127.27.18.254:34043)
0808 18:18:31.679608 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:31.679613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:31.679749 (+ 136us) server_negotiation.cc:407] Connection header received
0808 18:18:31.679910 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:31.679913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:31.679967 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:31.680060 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:31.680529 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.681056 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.681847 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.682083 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.682190 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:31.684858 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:58243: BlockingRecv error: recv got EOF from 127.27.18.194:58243 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 18:18:31 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:31 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:31.871582 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:31.865967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:57205)
0808 18:18:31.866290 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:31.866304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:31.866387 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:31.866592 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:31.866610 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:31.866840 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:31.867003 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:31.867011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.867874 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.867877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:31.868442 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:31.868468 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.868859 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.868862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:31.868921 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:31.869456 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:31.869468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:31.871474 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":153,"threads_started":1}
W20260808 18:18:31.871702 9820 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:31.866094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:57205 (local address 127.27.18.254:34043)
0808 18:18:31.866258 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:31.866262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:31.866351 (+ 89us) server_negotiation.cc:407] Connection header received
0808 18:18:31.866449 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:31.866451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:31.866494 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:31.866558 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:31.867119 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.867767 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.868560 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:31.868761 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:31.868844 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:31.871572 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:57205: BlockingRecv error: recv got EOF from 127.27.18.195:57205 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 18:18:32 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:32 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:32.513373 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:32.507844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:44721)
0808 18:18:32.508207 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:32.508230 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:32.508333 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:32.508647 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:32.508651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:32.508857 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:32.508954 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:32.508960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.509774 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.509779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:32.510331 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:32.510339 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.510800 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.510803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:32.510887 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:32.511395 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:32.511407 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:32.513194 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":91,"threads_started":1}
W20260808 18:18:32.513440 9827 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:32.507904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:44721 (local address 127.27.18.254:34043)
0808 18:18:32.508196 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:32.508204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:32.508310 (+ 106us) server_negotiation.cc:407] Connection header received
0808 18:18:32.508460 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:32.508462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:32.508510 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:32.508615 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:32.509078 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.509665 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.510454 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.510683 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.510793 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:32.513320 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:44721: BlockingRecv error: recv got EOF from 127.27.18.193:44721 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
Aug 08 18:18:32 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:32 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:32.691354 9828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:32.685747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:44645)
0808 18:18:32.686054 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:32.686067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:32.686162 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:32.686412 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:32.686414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:32.686590 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:32.686692 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:32.686697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.687484 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.687487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:32.688041 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:32.688066 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.688507 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.688510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:32.688571 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:32.689117 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:32.689129 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:32.691172 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":119,"threads_started":1}
W20260808 18:18:32.691475 9827 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:32.685884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:44645 (local address 127.27.18.254:34043)
0808 18:18:32.686044 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:32.686048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:32.686136 (+ 88us) server_negotiation.cc:407] Connection header received
0808 18:18:32.686241 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:32.686243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:32.686287 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:32.686370 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:32.686790 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.687336 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.688170 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.688395 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.688485 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:32.691342 (+ 2857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:44645: BlockingRecv error: recv got EOF from 127.27.18.194:44645 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 18:18:32 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:32 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:32.877871 9829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:32.872314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:55487)
0808 18:18:32.872612 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:32.872624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:32.872747 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:32.873044 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:32.873047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:32.873306 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:32.873430 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:32.873436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.874276 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.874281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:32.874835 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:32.874843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.875284 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.875287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:32.875364 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:32.875858 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:32.875870 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:32.877650 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":140,"threads_started":1}
W20260808 18:18:32.877890 9827 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:32.872359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:55487 (local address 127.27.18.254:34043)
0808 18:18:32.872529 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:32.872534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:32.872694 (+ 160us) server_negotiation.cc:407] Connection header received
0808 18:18:32.872865 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:32.872868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:32.872920 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:32.873022 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:32.873555 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.874172 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.874957 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:32.875177 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:32.875280 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:32.877788 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:55487: BlockingRecv error: recv got EOF from 127.27.18.195:55487 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 18:18:33 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:33 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:33.519936 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:33.514216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:52999)
0808 18:18:33.514523 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:33.514538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:33.514646 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:33.515074 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:33.515077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:33.515256 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:33.515348 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:33.515353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.516310 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.516327 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:33.516848 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:33.516856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.517362 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.517366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:33.517457 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:33.517916 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:33.517927 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:33.519768 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
W20260808 18:18:33.519989 9833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:33.514312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:52999 (local address 127.27.18.254:34043)
0808 18:18:33.514573 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:33.514577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:33.514589 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:33.514771 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:33.514774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:33.514839 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:33.514926 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:33.515512 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.516184 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.516995 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.517234 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.517354 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:33.519879 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:52999: BlockingRecv error: recv got EOF from 127.27.18.193:52999 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":107,"threads_started":1}
Aug 08 18:18:33 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:33 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:33.697566 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:33.692301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:33063)
0808 18:18:33.692571 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:33.692584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:33.692673 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:33.692909 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:33.692911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:33.693145 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:33.693306 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:33.693313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.694173 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.694176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:33.694681 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:33.694688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.695131 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.695134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:33.695212 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:33.695685 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:33.695704 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:33.697455 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
W20260808 18:18:33.697737 9833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:33.692356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:33063 (local address 127.27.18.254:34043)
0808 18:18:33.692497 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:33.692500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:33.692645 (+ 145us) server_negotiation.cc:407] Connection header received
0808 18:18:33.692745 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:33.692747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:33.692789 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:33.692860 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:33.693446 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.694071 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.694771 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.695005 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.695116 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:33.697583 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:33063: BlockingRecv error: recv got EOF from 127.27.18.194:33063 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 18:18:33 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:33 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:33.888330 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:33.878618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:35957)
0808 18:18:33.878943 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:33.878958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:33.879102 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:33.879390 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:33.879393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:33.879592 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:33.879685 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:33.879691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.880403 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.880408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:33.880923 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:33.880930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.881378 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.881381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:33.881460 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:33.886004 (+ 4544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:33.886016 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:33.888210 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":89,"threads_started":1}
W20260808 18:18:33.888458 9833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:33.878811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:35957 (local address 127.27.18.254:34043)
0808 18:18:33.878960 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:33.878964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:33.879039 (+ 75us) server_negotiation.cc:407] Connection header received
0808 18:18:33.879198 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:33.879201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:33.879249 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:33.879349 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:33.879797 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.880289 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.881036 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:33.881262 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:33.881373 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:33.888340 (+ 6967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:35957: BlockingRecv error: recv got EOF from 127.27.18.195:35957 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 18:18:34 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:34 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:34.526315 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:34.521099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:38403)
0808 18:18:34.521387 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:34.521400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:34.521500 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:34.521754 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:34.521757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:34.521937 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:34.522035 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:34.522040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.522803 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.522806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:34.523375 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:34.523382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.523836 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.523838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:34.523896 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:34.524455 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:34.524467 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:34.526128 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
W20260808 18:18:34.526511 9840 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:34.521163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:38403 (local address 127.27.18.254:34043)
0808 18:18:34.521399 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:34.521404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:34.521482 (+ 78us) server_negotiation.cc:407] Connection header received
0808 18:18:34.521583 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:34.521585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:34.521630 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:34.521709 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:34.522133 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.522697 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.523511 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.523733 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.523808 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:34.526341 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:38403: BlockingRecv error: recv got EOF from 127.27.18.193:38403 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":63,"threads_started":1}
Aug 08 18:18:34 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:34 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:34.704030 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:34.698488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:53329)
0808 18:18:34.698824 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:34.698837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:34.698959 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:34.699291 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:34.699294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:34.699596 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:34.699695 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:34.699701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.700416 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.700418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:34.700956 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:34.700963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.701383 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.701385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:34.701447 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:34.701950 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:34.701961 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:34.703907 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":145,"threads_started":1}
W20260808 18:18:34.704113 9840 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:34.698642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:53329 (local address 127.27.18.254:34043)
0808 18:18:34.698804 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:34.698809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:34.698918 (+ 109us) server_negotiation.cc:407] Connection header received
0808 18:18:34.699076 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:34.699081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:34.699138 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:34.699245 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:34.699805 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.700323 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.701062 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.701286 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.701363 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:34.704027 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:53329: BlockingRecv error: recv got EOF from 127.27.18.194:53329 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 18:18:34 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:34 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:34.894627 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:34.889097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:52697)
0808 18:18:34.889373 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:34.889387 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:34.889476 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:34.889717 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:34.889719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:34.889897 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:34.890011 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:34.890016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.890768 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.890771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:34.891343 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:34.891351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.891819 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.891824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:34.891908 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:34.892438 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:34.892450 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:34.894483 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":93,"threads_started":1}
W20260808 18:18:34.894742 9840 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:34.889251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:52697 (local address 127.27.18.254:34043)
0808 18:18:34.889406 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:34.889410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:34.889428 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:18:34.889556 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:34.889559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:34.889603 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:34.889659 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:34.890152 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.890673 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.891484 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:34.891709 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:34.891779 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:34.894612 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:52697: BlockingRecv error: recv got EOF from 127.27.18.195:52697 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 18:18:35 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:35 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:35.533103 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:35.527102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:33323)
0808 18:18:35.527434 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:35.527454 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:35.527600 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:35.527945 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:35.527948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:35.528169 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:35.528301 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:35.528324 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.529222 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.529227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:35.529872 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:35.529882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.530402 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.530406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:35.530496 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:35.531008 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:35.531021 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:35.532978 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":72,"threads_started":1}
W20260808 18:18:35.533401 9849 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:35.527225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:33323 (local address 127.27.18.254:34043)
0808 18:18:35.527585 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:35.527590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:35.527604 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:18:35.527731 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:35.527734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:35.527790 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:35.527890 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:35.528455 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.529086 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.530005 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.530273 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.530394 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:35.533140 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:33323: BlockingRecv error: recv got EOF from 127.27.18.193:33323 (error 108)
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":99,"threads_started":1}
Aug 08 18:18:35 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:35 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:35.710493 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:35.704972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:42565)
0808 18:18:35.705285 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:35.705303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:35.705448 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:35.705734 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:35.705737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:35.705966 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:35.706071 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:35.706077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.706802 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.706806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:35.707430 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:35.707444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.707920 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.707923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:35.708013 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:35.708502 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:35.708513 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:35.710327 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":89,"threads_started":1}
W20260808 18:18:35.710665 9849 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:35.705052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:42565 (local address 127.27.18.254:34043)
0808 18:18:35.705255 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:35.705260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:35.705385 (+ 125us) server_negotiation.cc:407] Connection header received
0808 18:18:35.705552 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:35.705556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:35.705609 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:35.705706 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:35.706187 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.706678 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.707575 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.707797 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.707902 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:35.710462 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:42565: BlockingRecv error: recv got EOF from 127.27.18.194:42565 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 08 18:18:35 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:35 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.195@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:35.900694 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:35.895411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.195:42151)
0808 18:18:35.895698 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:35.895710 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:35.895831 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:35.896097 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:35.896100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:35.896331 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:35.896426 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:35.896432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.897170 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.897173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:35.897762 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:35.897770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.898186 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.898188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:35.898248 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:35.898788 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:35.898799 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:35.900575 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":100,"threads_started":1}
W20260808 18:18:35.900866 9849 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:35.895445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:42151 (local address 127.27.18.254:34043)
0808 18:18:35.895604 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:35.895609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:35.895777 (+ 168us) server_negotiation.cc:407] Connection header received
0808 18:18:35.895935 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:35.895938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:35.895991 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:35.896069 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:35.896558 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.897061 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.897873 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:35.898087 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:35.898163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:35.900703 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.195:42151: BlockingRecv error: recv got EOF from 127.27.18.195:42151 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 18:18:36 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:36 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.193@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:36.542230 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:36.534132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.193:48349)
0808 18:18:36.534408 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:36.534420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:36.534510 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:36.534718 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:36.534720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:36.534879 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:36.534968 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:36.534973 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.535630 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.535632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:36.536112 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:36.536120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.536511 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.536513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:36.536570 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:36.540179 (+ 3609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:36.540194 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:36.542113 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
W20260808 18:18:36.542395 9853 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:36.534185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:48349 (local address 127.27.18.254:34043)
0808 18:18:36.534412 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:36.534415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:36.534489 (+ 74us) server_negotiation.cc:407] Connection header received
0808 18:18:36.534586 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:36.534588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:36.534627 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:36.534709 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:36.535058 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.535538 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.536215 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.536418 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.536486 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:36.542238 (+ 5752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.193:48349: BlockingRecv error: recv got EOF from 127.27.18.193:48349 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":82,"threads_started":1}
Aug 08 18:18:36 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 18:18:36 dist-test-slave-x4qh krb5kdc[8194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.18.194@KRBTEST.COM for oryx/127.27.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 18:18:36.717155 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:36.711586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34043 (local address 127.27.18.194:53387)
0808 18:18:36.711849 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:36.711873 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:36.711961 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:36.712253 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:36.712255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:36.712434 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:36.712585 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:36.712594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.713505 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.713508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:36.714229 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:36.714238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.714668 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.714670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:36.714730 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:36.715193 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:36.715204 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:36.717045 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.18.254:34043: Server oryx/127.27.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
W20260808 18:18:36.717289 9853 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:36.711649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:53387 (local address 127.27.18.254:34043)
0808 18:18:36.711804 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:36.711808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:36.711936 (+ 128us) server_negotiation.cc:407] Connection header received
0808 18:18:36.712060 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:36.712063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:36.712119 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:36.712199 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:36.712732 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.713395 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.714366 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:36.714577 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:36.714649 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:36.717168 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.18.194:53387: BlockingRecv error: recv got EOF from 127.27.18.194:53387 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:18:36.791517 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 9117
I20260808 18:18:36.802636 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 9342
I20260808 18:18:36.813366 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 9572
I20260808 18:18:36.823221 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 9002
2026-08-08T18:18:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18049 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T18:18:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:36Z Disabled control of system clock
I20260808 18:18:36.856469 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:46211
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:46211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 18:18:36.964232 9863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:36.964469 9863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:36.964501 9863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:36.968191 9863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:36.968236 9863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:36.968251 9863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:36.968267 9863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:36.968281 9863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:36.972582 9863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:46211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:46211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:36.973412 9863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:36.974581 9863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:36.982654 9871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:36.982692 9870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:36.982774 9863 server_base.cc:1034] running on GCE node
W20260808 18:18:36.982723 9874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:36.983337 9863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:36.984325 9863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:36.985494 9863 hybrid_clock.cc:648] HybridClock initialized: now 1786213116985467 us; error 38 us; skew 500 ppm
I20260808 18:18:36.987178 9863 webserver.cc:460] Webserver started at http://127.27.18.254:40661/ using document root <none> and password file <none>
I20260808 18:18:36.987790 9863 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:36.987860 9863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:36.988111 9863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:36.990056 9863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "4c65377f328042cc84afde74c13b343d"
format_stamp: "Formatted at 2026-08-08 18:18:36 on dist-test-slave-x4qh"
server_key: "9dbbb7209369f87dedae687b63c9969a"
server_key_iv: "c84030e84aef8d7075f722b84b561c84"
server_key_version: "encryptionkey@0"
I20260808 18:18:36.990618 9863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "4c65377f328042cc84afde74c13b343d"
format_stamp: "Formatted at 2026-08-08 18:18:36 on dist-test-slave-x4qh"
server_key: "9dbbb7209369f87dedae687b63c9969a"
server_key_iv: "c84030e84aef8d7075f722b84b561c84"
server_key_version: "encryptionkey@0"
I20260808 18:18:36.994194 9863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:18:36.996520 9883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:36.997617 9863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:36.997752 9863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4c65377f328042cc84afde74c13b343d"
format_stamp: "Formatted at 2026-08-08 18:18:36 on dist-test-slave-x4qh"
server_key: "9dbbb7209369f87dedae687b63c9969a"
server_key_iv: "c84030e84aef8d7075f722b84b561c84"
server_key_version: "encryptionkey@0"
I20260808 18:18:36.997864 9863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:37.019320 9863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:37.019954 9863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:37.020134 9863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:37.026953 9863 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:46211
I20260808 18:18:37.026975 9976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:46211 every 8 connection(s)
I20260808 18:18:37.027846 9863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:37.030712 9977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:37.032279 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 9863
I20260808 18:18:37.032418 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:37.032676 27723 external_mini_cluster.cc:1428] Setting key b7919d0ab943d257c784425149e3bcb0
I20260808 18:18:37.036738 9977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d: Bootstrap starting.
I20260808 18:18:37.038823 9977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:37.039608 9977 log.cc:824] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:37.041098 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.033855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.27.18.254:46211)
0808 18:18:37.034394 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:37.034408 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:37.034440 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:18:37.035066 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:37.035081 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:37.035464 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:37.035748 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:37.036145 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.037035 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.037934 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.038387 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.038553 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.039438 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:37.039446 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:37.039462 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:37.039880 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:37.039969 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:37.039997 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:37.040096 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:37.040253 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 18:18:37.040490 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
I20260808 18:18:37.041551 9977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d: No bootstrap required, opened a new log
I20260808 18:18:37.045368 9977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c65377f328042cc84afde74c13b343d" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46211 } }
I20260808 18:18:37.045586 9977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:37.045673 9977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c65377f328042cc84afde74c13b343d, State: Initialized, Role: FOLLOWER
I20260808 18:18:37.046028 9977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c65377f328042cc84afde74c13b343d" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46211 } }
I20260808 18:18:37.046166 9977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:37.046247 9977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:37.046346 9977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:37.047137 9977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c65377f328042cc84afde74c13b343d" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46211 } }
I20260808 18:18:37.047511 9977 leader_election.cc:302] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c65377f328042cc84afde74c13b343d; no voters:
I20260808 18:18:37.047804 9977 leader_election.cc:288] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:37.047906 9987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:37.048154 9987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [term 1 LEADER]: Becoming Leader. State: Replica: 4c65377f328042cc84afde74c13b343d, State: Running, Role: LEADER
I20260808 18:18:37.048496 9987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c65377f328042cc84afde74c13b343d" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46211 } }
I20260808 18:18:37.049014 9977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:37.050063 9989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c65377f328042cc84afde74c13b343d. Latest consensus state: current_term: 1 leader_uuid: "4c65377f328042cc84afde74c13b343d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c65377f328042cc84afde74c13b343d" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46211 } } }
I20260808 18:18:37.050201 9989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:37.050468 9988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c65377f328042cc84afde74c13b343d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c65377f328042cc84afde74c13b343d" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46211 } } }
I20260808 18:18:37.050550 9988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:37.050566 10002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:37.053335 10002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:37.058635 10002 catalog_manager.cc:1347] Generated new cluster ID: f36d8757465b46f0a274501f3f6845db
I20260808 18:18:37.058701 10002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:37.068815 10002 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:37.069849 10002 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:37.081370 10002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c65377f328042cc84afde74c13b343d: Generated new TSK 0
I20260808 18:18:37.081967 10002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:18:37.090389 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46211
--builtin_ntp_servers=127.27.18.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 18:18:37.209237 10013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:37.209461 10013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:37.209513 10013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:37.213532 10013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:37.213578 10013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:37.213626 10013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:37.217986 10013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:37.218906 10013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:37.220240 10013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:37.229434 10022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:37.229492 10013 server_base.cc:1034] running on GCE node
W20260808 18:18:37.229444 10020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:37.229441 10024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:37.229943 10013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:37.230543 10013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:37.231717 10013 hybrid_clock.cc:648] HybridClock initialized: now 1786213117231709 us; error 26 us; skew 500 ppm
I20260808 18:18:37.233492 10013 webserver.cc:460] Webserver started at http://127.27.18.193:36519/ using document root <none> and password file <none>
I20260808 18:18:37.234025 10013 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:37.234109 10013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:37.234328 10013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:37.236300 10013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "54fdb6e8a4334c91ba89fe6fd211b0ea"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "40e426a181ef2f6edd11d7a31a41c02d"
server_key_iv: "17223301a2b56f546d1b7ee19165ce08"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.236855 10013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "54fdb6e8a4334c91ba89fe6fd211b0ea"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "40e426a181ef2f6edd11d7a31a41c02d"
server_key_iv: "17223301a2b56f546d1b7ee19165ce08"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.240337 10013 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:37.242574 10032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.243709 10013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:37.243858 10013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "54fdb6e8a4334c91ba89fe6fd211b0ea"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "40e426a181ef2f6edd11d7a31a41c02d"
server_key_iv: "17223301a2b56f546d1b7ee19165ce08"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.243959 10013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:37.255678 10013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:37.256223 10013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:37.256407 10013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:37.256934 10013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:37.257884 10013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:37.257951 10013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.258015 10013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:37.258064 10013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.266621 10013 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:32921
I20260808 18:18:37.266650 10213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:32921 every 8 connection(s)
I20260808 18:18:37.267511 10013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:37.268062 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 10013
I20260808 18:18:37.268168 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:37.268474 27723 external_mini_cluster.cc:1428] Setting key 6ace0c8babc50544f73bfd89306bea07
I20260808 18:18:37.270370 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46211
--builtin_ntp_servers=127.27.18.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 18:18:37.276729 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.269239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36109 (local address 127.27.18.254:46211)
0808 18:18:37.269370 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:37.269373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:37.270508 (+ 1135us) server_negotiation.cc:407] Connection header received
0808 18:18:37.272025 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:37.272028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:37.272087 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:37.272137 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:37.273125 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.273753 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.274810 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.275037 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.275146 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.276253 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:37.276256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:37.276258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:37.276269 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:37.276286 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:37.276294 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:37.276374 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:37.276533 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 18:18:37.276595 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:18:37.277489 10224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.269675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46211 (local address 127.27.18.193:36109)
0808 18:18:37.270369 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:37.270418 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:37.271795 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:37.272436 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:37.272442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:37.272513 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:37.272971 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:37.272983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.273872 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.273875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:37.274684 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:37.274693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.275188 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.275192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:37.275347 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.276001 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:37.276002 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:37.276038 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:37.276095 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:37.276377 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:37.276389 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:37.276395 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:18:37.276519 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 18:18:37.276761 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":134,"threads_started":1}
I20260808 18:18:37.278687 10214 heartbeater.cc:347] Connected to a master server at 127.27.18.254:46211
I20260808 18:18:37.278910 10214 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:37.279415 10214 heartbeater.cc:511] Master 127.27.18.254:46211 requested a full tablet report, sending...
I20260808 18:18:37.281005 9919 ts_manager.cc:194] Registered new tserver with Master: 54fdb6e8a4334c91ba89fe6fd211b0ea (127.27.18.193:32921)
I20260808 18:18:37.282655 9919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.193:36109
W20260808 18:18:37.380407 10225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:37.380645 10225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:37.380686 10225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:37.384356 10225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:37.384415 10225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:37.384462 10225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:37.389062 10225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:37.389925 10225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:37.391099 10225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:37.400635 10241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:37.400699 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:37.400648 10239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:37.400962 10225 server_base.cc:1034] running on GCE node
I20260808 18:18:37.401407 10225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:37.402024 10225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:37.403216 10225 hybrid_clock.cc:648] HybridClock initialized: now 1786213117403208 us; error 41 us; skew 500 ppm
I20260808 18:18:37.405018 10225 webserver.cc:460] Webserver started at http://127.27.18.194:36891/ using document root <none> and password file <none>
I20260808 18:18:37.405584 10225 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:37.405678 10225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:37.405887 10225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:37.407939 10225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "ed22ae4937174c73b1f83d51a5722584"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "57081643999fb4ff6c0ce04f35679639"
server_key_iv: "30e731e1eab2f1fb82c5d57a929baa16"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.408504 10225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "ed22ae4937174c73b1f83d51a5722584"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "57081643999fb4ff6c0ce04f35679639"
server_key_iv: "30e731e1eab2f1fb82c5d57a929baa16"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.411993 10225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:37.414422 10250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.415545 10225 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:37.415681 10225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "ed22ae4937174c73b1f83d51a5722584"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "57081643999fb4ff6c0ce04f35679639"
server_key_iv: "30e731e1eab2f1fb82c5d57a929baa16"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.415791 10225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:37.428117 10225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:37.428699 10225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:37.428877 10225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:37.429411 10225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:37.430219 10225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:37.430284 10225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.430347 10225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:37.430394 10225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.439057 10225 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:35841
I20260808 18:18:37.439101 10419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:35841 every 8 connection(s)
I20260808 18:18:37.439913 10225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:37.447800 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 10225
I20260808 18:18:37.447916 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:37.448213 27723 external_mini_cluster.cc:1428] Setting key 7d223c69b3b59ed54626ca651f4dbc13
I20260808 18:18:37.448504 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.441555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:37005 (local address 127.27.18.254:46211)
0808 18:18:37.441693 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:37.441697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:37.442622 (+ 925us) server_negotiation.cc:407] Connection header received
0808 18:18:37.443942 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:37.443945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:37.443991 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:37.444063 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:37.444865 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.445366 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.446522 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.446796 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.446877 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.448010 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:37.448013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:37.448015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:37.448027 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:37.448043 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:37.448050 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:37.448130 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:37.448291 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 18:18:37.448355 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:18:37.449515 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.441897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46211 (local address 127.27.18.194:37005)
0808 18:18:37.442490 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:37.442527 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:37.443768 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:37.444227 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:37.444232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:37.444301 (+ 69us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:37.444744 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:37.444756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.445487 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.445490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:37.446374 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:37.446383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.446922 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.446925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:37.447052 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.447708 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:37.447710 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:37.447745 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:37.447834 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:37.448133 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:37.448145 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:37.448153 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:18:37.448288 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 18:18:37.448562 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
I20260808 18:18:37.450253 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46211
--builtin_ntp_servers=127.27.18.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 18:18:37.450752 10420 heartbeater.cc:347] Connected to a master server at 127.27.18.254:46211
I20260808 18:18:37.450977 10420 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:37.451545 10420 heartbeater.cc:511] Master 127.27.18.254:46211 requested a full tablet report, sending...
I20260808 18:18:37.452622 9919 ts_manager.cc:194] Registered new tserver with Master: ed22ae4937174c73b1f83d51a5722584 (127.27.18.194:35841)
I20260808 18:18:37.453332 9919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.194:37005
W20260808 18:18:37.561165 10425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:37.561378 10425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:37.561412 10425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:37.565241 10425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:37.565285 10425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:37.565339 10425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:37.569823 10425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:37.570652 10425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:37.571848 10425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:37.580596 10435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:37.580612 10436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:37.580778 10438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:37.580865 10425 server_base.cc:1034] running on GCE node
I20260808 18:18:37.581302 10425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:37.581902 10425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:37.583084 10425 hybrid_clock.cc:648] HybridClock initialized: now 1786213117583063 us; error 35 us; skew 500 ppm
I20260808 18:18:37.584842 10425 webserver.cc:460] Webserver started at http://127.27.18.195:39801/ using document root <none> and password file <none>
I20260808 18:18:37.585362 10425 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:37.585421 10425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:37.585655 10425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:37.587565 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "263b2dd3c7c346fd97e52004ba20736d"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "53bcd2661ef5ece410f8271faea267c6"
server_key_iv: "c59ff54159aa042bf094b1766397e468"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.588086 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "263b2dd3c7c346fd97e52004ba20736d"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "53bcd2661ef5ece410f8271faea267c6"
server_key_iv: "c59ff54159aa042bf094b1766397e468"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.591325 10425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:37.593531 10450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.594561 10425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:18:37.594694 10425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "263b2dd3c7c346fd97e52004ba20736d"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "53bcd2661ef5ece410f8271faea267c6"
server_key_iv: "c59ff54159aa042bf094b1766397e468"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.594800 10425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:37.609941 10425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:37.610507 10425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:37.610682 10425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:37.611181 10425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:37.612069 10425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:37.612139 10425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.612206 10425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:37.612247 10425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.621482 10425 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:33629
I20260808 18:18:37.621506 10614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:33629 every 8 connection(s)
I20260808 18:18:37.622241 10425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:37.627664 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 10425
I20260808 18:18:37.627779 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:37.628070 27723 external_mini_cluster.cc:1428] Setting key 7996f84c34dfc6ce3ad20d3584884dec
I20260808 18:18:37.630937 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.623833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:37177 (local address 127.27.18.254:46211)
0808 18:18:37.623955 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:37.623958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:37.624891 (+ 933us) server_negotiation.cc:407] Connection header received
0808 18:18:37.626087 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:37.626090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:37.626133 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:37.626200 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:37.626955 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.627531 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.629066 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.629347 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.629426 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.630534 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:37.630537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:37.630539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:37.630550 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:37.630568 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:37.630576 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:37.630632 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:37.630784 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 18:18:37.630837 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:18:37.631762 10617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.624145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46211 (local address 127.27.18.195:37177)
0808 18:18:37.624758 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:37.624793 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:37.625910 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:37.626343 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:37.626349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:37.626417 (+ 68us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:37.626821 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:37.626832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.627669 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.627688 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:37.628962 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:37.628974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:37.629488 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:37.629492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:37.629639 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:37.630244 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:37.630247 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:37.630313 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:37.630369 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:37.630651 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:37.630658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:37.630662 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:18:37.630762 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 18:18:37.630970 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":99,"threads_started":1}
I20260808 18:18:37.632745 10615 heartbeater.cc:347] Connected to a master server at 127.27.18.254:46211
I20260808 18:18:37.632956 10615 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:37.633419 10615 heartbeater.cc:511] Master 127.27.18.254:46211 requested a full tablet report, sending...
I20260808 18:18:37.634331 9919 ts_manager.cc:194] Registered new tserver with Master: 263b2dd3c7c346fd97e52004ba20736d (127.27.18.195:33629)
I20260808 18:18:37.634840 9919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.195:37177
I20260808 18:18:37.641587 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:18:37.644085 9985 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:37.643183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.27.18.254:46211)
0808 18:18:37.643315 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:37.643318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:37.643541 (+ 223us) server_negotiation.cc:407] Connection header received
0808 18:18:37.643622 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:37.643624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:37.643659 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:37.643719 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:37.643941 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57914: BlockingRecv error: recv got EOF from 127.0.0.1:57914 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:18:37.644200 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 10013
I20260808 18:18:37.661805 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 10225
I20260808 18:18:37.669426 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 10425
I20260808 18:18:37.684954 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 9863
2026-08-08T18:18:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (865 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T18:18:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:37Z Disabled control of system clock
I20260808 18:18:37.722062 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36379
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:36379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:18:37.826309 10630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:37.826540 10630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:37.826589 10630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:37.830322 10630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:37.830369 10630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:37.830384 10630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:37.830415 10630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:37.830430 10630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:37.834532 10630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:36379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:37.835330 10630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:37.836572 10630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:37.844559 10641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:37.844678 10637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:37.844657 10630 server_base.cc:1034] running on GCE node
W20260808 18:18:37.844570 10638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:37.845280 10630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:37.846295 10630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:37.847452 10630 hybrid_clock.cc:648] HybridClock initialized: now 1786213117847441 us; error 26 us; skew 500 ppm
I20260808 18:18:37.849040 10630 webserver.cc:460] Webserver started at http://127.27.18.254:39779/ using document root <none> and password file <none>
I20260808 18:18:37.849519 10630 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:37.849601 10630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:37.849805 10630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:37.851707 10630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "eaf259d82f51642d0e7a4cc002da6122"
server_key_iv: "29d85074b713c5f42a7121fd727d8b72"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.852211 10630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "eaf259d82f51642d0e7a4cc002da6122"
server_key_iv: "29d85074b713c5f42a7121fd727d8b72"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.855518 10630 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:37.857694 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:37.858789 10630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:37.858922 10630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3"
format_stamp: "Formatted at 2026-08-08 18:18:37 on dist-test-slave-x4qh"
server_key: "eaf259d82f51642d0e7a4cc002da6122"
server_key_iv: "29d85074b713c5f42a7121fd727d8b72"
server_key_version: "encryptionkey@0"
I20260808 18:18:37.859027 10630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:37.863279 10630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:37.863863 10630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:37.864044 10630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:37.870328 10630 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:36379
I20260808 18:18:37.870338 10727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:36379 every 8 connection(s)
I20260808 18:18:37.871075 10630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:37.873977 10728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:37.877728 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 10630
I20260808 18:18:37.877848 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:37.878149 27723 external_mini_cluster.cc:1428] Setting key c0d873f2057b4e07245066ea28f04b08
I20260808 18:18:37.879320 10728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3: Bootstrap starting.
I20260808 18:18:37.881270 10728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:37.882205 10728 log.cc:824] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:37.882903 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:37.879473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.27.18.254:36379)
0808 18:18:37.880023 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:37.880035 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:37.880065 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:18:37.880597 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:37.880611 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:37.880904 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:37.881190 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:37.881814 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:37.881818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:37.881828 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:37.881836 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:37.881879 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:37.881892 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:37.882005 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:37.882114 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 18:18:37.882347 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":118,"threads_started":1}
I20260808 18:18:37.884039 10728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3: No bootstrap required, opened a new log
I20260808 18:18:37.887323 10728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36379 } }
I20260808 18:18:37.887579 10728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:37.887665 10728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0edb6be0ba4e25a795d9e6bb12b7a3, State: Initialized, Role: FOLLOWER
I20260808 18:18:37.888115 10728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36379 } }
I20260808 18:18:37.888254 10728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:37.888314 10728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:37.888410 10728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:37.889310 10728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36379 } }
I20260808 18:18:37.889636 10728 leader_election.cc:302] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e0edb6be0ba4e25a795d9e6bb12b7a3; no voters:
I20260808 18:18:37.889931 10728 leader_election.cc:288] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:37.890046 10734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:37.890292 10734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [term 1 LEADER]: Becoming Leader. State: Replica: 1e0edb6be0ba4e25a795d9e6bb12b7a3, State: Running, Role: LEADER
I20260808 18:18:37.890564 10734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36379 } }
I20260808 18:18:37.891047 10728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:37.892104 10735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36379 } } }
I20260808 18:18:37.892261 10735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:37.892570 10738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e0edb6be0ba4e25a795d9e6bb12b7a3. Latest consensus state: current_term: 1 leader_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e0edb6be0ba4e25a795d9e6bb12b7a3" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36379 } } }
I20260808 18:18:37.892655 10738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:37.892817 10749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:37.895601 10749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:37.900655 10749 catalog_manager.cc:1347] Generated new cluster ID: dcaca037e5ba498e9e724a65d4ca0f6a
I20260808 18:18:37.900744 10749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:37.919008 10749 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:37.919291 10749 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:37.929550 10749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e0edb6be0ba4e25a795d9e6bb12b7a3: Generated new TSK 0
I20260808 18:18:37.930119 10749 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:18:37.934314 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36379
--builtin_ntp_servers=127.27.18.212:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:18:38.053598 10765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:38.053829 10765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:38.053866 10765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:38.057605 10765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:38.057647 10765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:38.057725 10765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:38.062083 10765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:38.062994 10765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:38.064213 10765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:38.072901 10782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:38.072892 10778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:38.072896 10780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:38.073192 10765 server_base.cc:1034] running on GCE node
I20260808 18:18:38.073580 10765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:38.074165 10765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:38.075356 10765 hybrid_clock.cc:648] HybridClock initialized: now 1786213118075334 us; error 30 us; skew 500 ppm
I20260808 18:18:38.076920 10765 webserver.cc:460] Webserver started at http://127.27.18.193:46303/ using document root <none> and password file <none>
I20260808 18:18:38.077478 10765 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:38.077570 10765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:38.077791 10765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:38.079829 10765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "c02524d37d054103a1433c7c3448b417"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "35901087fb7cb1860a2601fedb681c66"
server_key_iv: "038785b8b3032c0069350a9d47100ee7"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.080353 10765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "c02524d37d054103a1433c7c3448b417"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "35901087fb7cb1860a2601fedb681c66"
server_key_iv: "038785b8b3032c0069350a9d47100ee7"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.083606 10765 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:18:38.085815 10791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.086827 10765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:38.086964 10765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "c02524d37d054103a1433c7c3448b417"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "35901087fb7cb1860a2601fedb681c66"
server_key_iv: "038785b8b3032c0069350a9d47100ee7"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.087070 10765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:38.091195 10765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:38.092034 10765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:38.092279 10765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:38.092828 10765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:38.093756 10765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:38.093829 10765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.093897 10765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:38.093945 10765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.102694 10765 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:38441
I20260808 18:18:38.102715 10962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:38441 every 8 connection(s)
I20260808 18:18:38.103519 10765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:38.108892 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:38.105192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:40233 (local address 127.27.18.254:36379)
0808 18:18:38.105342 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:38.105352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:38.106231 (+ 879us) server_negotiation.cc:407] Connection header received
0808 18:18:38.107413 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:38.107416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:38.107483 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:38.107558 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:38.108510 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:38.108512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:38.108514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:38.108526 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:38.108546 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:38.108565 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:38.108633 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:38.108757 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 18:18:38.108791 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:18:38.109639 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:38.105489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36379 (local address 127.27.18.193:40233)
0808 18:18:38.106087 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:38.106122 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:38.107201 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:38.107673 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:38.107679 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:38.107756 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:38.108269 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:38.108271 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:38.108306 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:38.108368 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:38.108644 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:38.108651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:38.108656 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:18:38.108771 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 18:18:38.108956 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
I20260808 18:18:38.110819 10963 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36379
I20260808 18:18:38.111042 10963 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:38.111497 10963 heartbeater.cc:511] Master 127.27.18.254:36379 requested a full tablet report, sending...
I20260808 18:18:38.111948 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 10765
I20260808 18:18:38.112075 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:38.112378 27723 external_mini_cluster.cc:1428] Setting key 1fba3aadd1569bac200c2bd4f142364c
I20260808 18:18:38.113071 10677 ts_manager.cc:194] Registered new tserver with Master: c02524d37d054103a1433c7c3448b417 (127.27.18.193:38441)
I20260808 18:18:38.114382 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36379
--builtin_ntp_servers=127.27.18.212:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 18:18:38.221370 10967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:38.221580 10967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:38.221645 10967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:38.225525 10967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:38.225579 10967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:38.225673 10967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:38.230008 10967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:38.230943 10967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:38.232167 10967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:38.240949 10979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:38.240935 10984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:38.241047 10980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:38.241433 10967 server_base.cc:1034] running on GCE node
I20260808 18:18:38.241786 10967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:38.242316 10967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:38.243444 10967 hybrid_clock.cc:648] HybridClock initialized: now 1786213118243416 us; error 39 us; skew 500 ppm
I20260808 18:18:38.245188 10967 webserver.cc:460] Webserver started at http://127.27.18.194:38775/ using document root <none> and password file <none>
I20260808 18:18:38.245711 10967 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:38.245774 10967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:38.246013 10967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:38.247956 10967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "80cd81f296854ea3910c8d0944e0211c"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "d9791f7a8ac27631f359b45403b9890c"
server_key_iv: "c53c7af713c48f18c24f0bfc7fa3c874"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.248482 10967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "80cd81f296854ea3910c8d0944e0211c"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "d9791f7a8ac27631f359b45403b9890c"
server_key_iv: "c53c7af713c48f18c24f0bfc7fa3c874"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.251796 10967 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:18:38.253911 10993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.254936 10967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:18:38.255070 10967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "80cd81f296854ea3910c8d0944e0211c"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "d9791f7a8ac27631f359b45403b9890c"
server_key_iv: "c53c7af713c48f18c24f0bfc7fa3c874"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.255188 10967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:38.259212 10967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:38.259789 10967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:38.259969 10967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:38.260459 10967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:38.261302 10967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:38.261365 10967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.261422 10967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:38.261471 10967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.270262 10967 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:39559
I20260808 18:18:38.270287 11167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:39559 every 8 connection(s)
I20260808 18:18:38.271021 10967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:18:38.271586 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 10967
I20260808 18:18:38.271734 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:38.272014 27723 external_mini_cluster.cc:1428] Setting key f3533550a0e85c1bd9739e7e2993a326
I20260808 18:18:38.274250 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36379
--builtin_ntp_servers=127.27.18.212:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 18:18:38.277045 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:38.272710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:52095 (local address 127.27.18.254:36379)
0808 18:18:38.272838 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:38.272858 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:38.274044 (+ 1186us) server_negotiation.cc:407] Connection header received
0808 18:18:38.275331 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:38.275334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:38.275399 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:38.275471 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:38.276567 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:38.276570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:38.276572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:38.276584 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:38.276605 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:38.276634 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:38.276726 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:38.276906 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 18:18:38.276944 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:18:38.278067 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:38.273166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36379 (local address 127.27.18.194:52095)
0808 18:18:38.273907 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:38.273945 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:38.275140 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:38.275614 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:38.275619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:38.275692 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:38.276265 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:38.276267 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:38.276319 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:38.276405 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:38.276750 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:38.276759 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:38.276767 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:18:38.276937 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 18:18:38.277164 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":172,"threads_started":1}
I20260808 18:18:38.279170 11168 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36379
I20260808 18:18:38.279425 11168 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:38.279785 11168 heartbeater.cc:511] Master 127.27.18.254:36379 requested a full tablet report, sending...
I20260808 18:18:38.280790 10677 ts_manager.cc:194] Registered new tserver with Master: 80cd81f296854ea3910c8d0944e0211c (127.27.18.194:39559)
W20260808 18:18:38.382817 11172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:38.383054 11172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:38.383119 11172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:38.387033 11172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:38.387109 11172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:38.387189 11172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:38.392256 11172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:38.393280 11172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:38.394573 11172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:38.403679 11179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:38.403667 11180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:38.403798 11183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:38.403992 11172 server_base.cc:1034] running on GCE node
I20260808 18:18:38.404419 11172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:38.405009 11172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:38.406169 11172 hybrid_clock.cc:648] HybridClock initialized: now 1786213118406155 us; error 24 us; skew 500 ppm
I20260808 18:18:38.407999 11172 webserver.cc:460] Webserver started at http://127.27.18.195:42907/ using document root <none> and password file <none>
I20260808 18:18:38.408535 11172 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:38.408607 11172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:38.408825 11172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:38.410686 11172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "1308f779fb914d9f88913b735e9e6f2d"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "f6ef091ab6497aafa975abe87dbf6d18"
server_key_iv: "63d1004d5470310d4cbf56ed48571066"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.411234 11172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "1308f779fb914d9f88913b735e9e6f2d"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "f6ef091ab6497aafa975abe87dbf6d18"
server_key_iv: "63d1004d5470310d4cbf56ed48571066"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.414554 11172 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 18:18:38.416790 11190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.417826 11172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:18:38.417943 11172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "1308f779fb914d9f88913b735e9e6f2d"
format_stamp: "Formatted at 2026-08-08 18:18:38 on dist-test-slave-x4qh"
server_key: "f6ef091ab6497aafa975abe87dbf6d18"
server_key_iv: "63d1004d5470310d4cbf56ed48571066"
server_key_version: "encryptionkey@0"
I20260808 18:18:38.418043 11172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:38.422358 11172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:38.422888 11172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:38.423066 11172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:38.423681 11172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:38.424530 11172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:38.424575 11172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.424640 11172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:38.424677 11172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:38.432962 11172 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:43035
I20260808 18:18:38.433024 11371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:43035 every 8 connection(s)
I20260808 18:18:38.433727 11172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:18:38.439347 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:38.435360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:36383 (local address 127.27.18.254:36379)
0808 18:18:38.435519 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:38.435522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:38.436443 (+ 921us) server_negotiation.cc:407] Connection header received
0808 18:18:38.437751 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:38.437755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:38.437813 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:38.437914 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:38.438877 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:38.438880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:38.438883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:18:38.438894 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:38.438916 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:18:38.438930 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:38.439016 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:38.439162 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 18:18:38.439221 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 18:18:38.440073 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:38.435744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36379 (local address 127.27.18.195:36383)
0808 18:18:38.436314 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:38.436348 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:38.437552 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:38.438067 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:38.438073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:38.438149 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:38.438650 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:18:38.438651 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:38.438684 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:18:38.438741 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:38.439014 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:38.439023 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:38.439031 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 18:18:38.439207 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 18:18:38.439468 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260808 18:18:38.441087 11372 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36379
I20260808 18:18:38.441277 11372 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:38.441656 11372 heartbeater.cc:511] Master 127.27.18.254:36379 requested a full tablet report, sending...
I20260808 18:18:38.441663 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 11172
I20260808 18:18:38.441766 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:38.442050 27723 external_mini_cluster.cc:1428] Setting key dcc523309c635085835f81c257954732
I20260808 18:18:38.442656 10677 ts_manager.cc:194] Registered new tserver with Master: 1308f779fb914d9f88913b735e9e6f2d (127.27.18.195:43035)
I20260808 18:18:38.443958 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 18:18:38.446450 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:18:38.445707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.27.18.254:36379)
0808 18:18:38.445848 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:38.445851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:38.446029 (+ 178us) server_negotiation.cc:407] Connection header received
0808 18:18:38.446101 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:38.446102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:38.446127 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:38.446182 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:38.446356 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34626: BlockingRecv error: recv got EOF from 127.0.0.1:34626 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:38.446588 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 10765
I20260808 18:18:38.462571 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 10967
I20260808 18:18:38.471499 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 11172
I20260808 18:18:38.479761 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 10630
2026-08-08T18:18:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (792 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:38 dist-test-slave-x4qh krb5kdc[11385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:38 dist-test-slave-x4qh krb5kdc[11385](info): set up 2 sockets
Aug 08 18:18:38 dist-test-slave-x4qh krb5kdc[11385](info): commencing operation
krb5kdc: starting...
W20260808 18:18:39.683565 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:39 dist-test-slave-x4qh krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:18:39.799284 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41953
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:41953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:39.905467 11408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:39.905707 11408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:39.905783 11408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:39.909662 11408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:18:39.909713 11408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:39.909732 11408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:39.909747 11408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:18:39.909762 11408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:18:39.914597 11408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:41953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:39.915570 11408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:39.916505 11408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:39.924643 11417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:39.924729 11408 server_base.cc:1034] running on GCE node
W20260808 18:18:39.924674 11418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:39.924674 11420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:39.925467 11408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:39.926384 11408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:39.927526 11408 hybrid_clock.cc:648] HybridClock initialized: now 1786213119927507 us; error 27 us; skew 500 ppm
Aug 08 18:18:39 dist-test-slave-x4qh krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213119, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:39.932807 11408 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:18:39.933595 11408 webserver.cc:460] Webserver started at http://127.27.18.254:45153/ using document root <none> and password file <none>
I20260808 18:18:39.934106 11408 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:39.934176 11408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:39.934345 11408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:39.936247 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "5028adf0102d44c2898f0fc725045f08"
format_stamp: "Formatted at 2026-08-08 18:18:39 on dist-test-slave-x4qh"
server_key: "595219daf8fd2328d9ba58e262a8bd8e"
server_key_iv: "751314f11da5d552d0d29e563d7e8c77"
server_key_version: "encryptionkey@0"
I20260808 18:18:39.936745 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "5028adf0102d44c2898f0fc725045f08"
format_stamp: "Formatted at 2026-08-08 18:18:39 on dist-test-slave-x4qh"
server_key: "595219daf8fd2328d9ba58e262a8bd8e"
server_key_iv: "751314f11da5d552d0d29e563d7e8c77"
server_key_version: "encryptionkey@0"
I20260808 18:18:39.940179 11408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:39.942405 11428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:39.943650 11408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:39.943778 11408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5028adf0102d44c2898f0fc725045f08"
format_stamp: "Formatted at 2026-08-08 18:18:39 on dist-test-slave-x4qh"
server_key: "595219daf8fd2328d9ba58e262a8bd8e"
server_key_iv: "751314f11da5d552d0d29e563d7e8c77"
server_key_version: "encryptionkey@0"
I20260808 18:18:39.943885 11408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:39.955878 11408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:39.958304 11408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:39.958506 11408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:39.965286 11408 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:41953
I20260808 18:18:39.965319 11507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:41953 every 8 connection(s)
I20260808 18:18:39.966144 11408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:18:39.968755 11509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:39.973798 11509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08: Bootstrap starting.
I20260808 18:18:39.974766 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 11408
I20260808 18:18:39.974869 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:18:39.975149 27723 external_mini_cluster.cc:1428] Setting key 737833f0d2d70902f39072c8488297a4
I20260808 18:18:39.976264 11509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:39.976949 11509 log.cc:824] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:39.978688 11509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08: No bootstrap required, opened a new log
I20260808 18:18:39.981909 11509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5028adf0102d44c2898f0fc725045f08" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41953 } }
I20260808 18:18:39.982127 11509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:39.982208 11509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5028adf0102d44c2898f0fc725045f08, State: Initialized, Role: FOLLOWER
I20260808 18:18:39.982645 11509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5028adf0102d44c2898f0fc725045f08" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41953 } }
I20260808 18:18:39.982815 11509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:18:39.982895 11509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:18:39.982988 11509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:18:39 dist-test-slave-x4qh krb5kdc[11385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:39.984047 11509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5028adf0102d44c2898f0fc725045f08" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41953 } }
I20260808 18:18:39.984447 11509 leader_election.cc:302] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5028adf0102d44c2898f0fc725045f08; no voters:
I20260808 18:18:39.984800 11509 leader_election.cc:288] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:18:39.984917 11514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:39.985133 11514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [term 1 LEADER]: Becoming Leader. State: Replica: 5028adf0102d44c2898f0fc725045f08, State: Running, Role: LEADER
I20260808 18:18:39.985436 11514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5028adf0102d44c2898f0fc725045f08" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41953 } }
I20260808 18:18:39.986084 11509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:18:39.986518 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5028adf0102d44c2898f0fc725045f08. Latest consensus state: current_term: 1 leader_uuid: "5028adf0102d44c2898f0fc725045f08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5028adf0102d44c2898f0fc725045f08" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41953 } } }
I20260808 18:18:39.986640 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:39.987097 11519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5028adf0102d44c2898f0fc725045f08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5028adf0102d44c2898f0fc725045f08" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41953 } } }
I20260808 18:18:39.987237 11519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08 [sys.catalog]: This master's current role is: LEADER
I20260808 18:18:39.987632 11523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:18:39.989933 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:39.976157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.27.18.254:41953)
0808 18:18:39.976549 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:39.976562 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:39.976593 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:18:39.977206 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:39.977239 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:39.977648 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:39.977979 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:39.978776 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:39.979859 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:39.980697 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:39.981157 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:39.981330 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:39.983988 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:39.983994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:39.984005 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:39.984030 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:39.986292 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:39.986844 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:39.986850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:39.986856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:39.987010 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:39.987398 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:39.987401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:39.987403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:39.987892 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:39.988643 (+ 751us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:39.988783 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 18:18:39.989094 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":133,"threads_started":1}
I20260808 18:18:39.991457 11523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:18:39.996452 11523 catalog_manager.cc:1347] Generated new cluster ID: 77394de4e2f0484fb20642154d4d7f1c
I20260808 18:18:39.996510 11523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:18:40.029109 11523 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:18:40.029948 11523 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:18:40.035890 11523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08: Generated new TSK 0
I20260808 18:18:40.036476 11523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:18:40.066594 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41953
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:40.171564 11548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:40.171756 11548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:40.171793 11548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:40.175448 11548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:40.175505 11548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:40.175595 11548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:18:40.180519 11548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:40.181372 11548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:40.182569 11548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:40.191535 11555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:40.191519 11559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:40.191519 11554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:40.191659 11548 server_base.cc:1034] running on GCE node
I20260808 18:18:40.192250 11548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:40.192809 11548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:40.193989 11548 hybrid_clock.cc:648] HybridClock initialized: now 1786213120193976 us; error 33 us; skew 500 ppm
Aug 08 18:18:40 dist-test-slave-x4qh krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213120, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:40.196559 11548 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:18:40.197302 11548 webserver.cc:460] Webserver started at http://127.27.18.193:34449/ using document root <none> and password file <none>
I20260808 18:18:40.197816 11548 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:40.197904 11548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:40.198122 11548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:40.200049 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "35d6d2c848331ae6bc67865556abdd60"
server_key_iv: "f4cb5a86342572ea6e2d4e76cf37fec6"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.200575 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "35d6d2c848331ae6bc67865556abdd60"
server_key_iv: "f4cb5a86342572ea6e2d4e76cf37fec6"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.204061 11548 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:40.206465 11569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.207664 11548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:40.207818 11548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "35d6d2c848331ae6bc67865556abdd60"
server_key_iv: "f4cb5a86342572ea6e2d4e76cf37fec6"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.207926 11548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:40.225811 11548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:40.228437 11548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:40.228622 11548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:40.229168 11548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:40.230178 11548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:40.230247 11548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.230316 11548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:40.230355 11548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.239264 11548 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:43713
I20260808 18:18:40.239289 11747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:43713 every 8 connection(s)
I20260808 18:18:40.240104 11548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:18:40.242336 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 11548
I20260808 18:18:40.242468 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:18:40.242795 27723 external_mini_cluster.cc:1428] Setting key 1ffcf8e2621930cc964dac7f7c81f74a
Aug 08 18:18:40 dist-test-slave-x4qh krb5kdc[11385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213120, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:18:40.253713 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.241750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36999 (local address 127.27.18.254:41953)
0808 18:18:40.241893 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.241898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.242742 (+ 844us) server_negotiation.cc:407] Connection header received
0808 18:18:40.244448 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.244452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.244507 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.244608 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:40.245742 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.246254 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.247564 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.247811 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.247960 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.250313 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:40.250316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:40.250318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:40.250342 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:40.251808 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.252291 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.252294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.252296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.252374 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.252690 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.252695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.252699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.252903 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:40.253005 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.253432 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.253547 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:18:40.254316 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.242136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41953 (local address 127.27.18.193:36999)
0808 18:18:40.242584 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.242633 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.244226 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.244778 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.244784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.245144 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:40.245596 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.245608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.246381 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.246385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.247418 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.247430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.247980 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.247984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.248166 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.248745 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:40.248759 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:40.250146 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:40.251929 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:40.251935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:40.251947 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:40.252190 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:40.252466 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:40.252469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:40.252471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:40.252597 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:40.253005 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:40.253012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:40.253222 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.253391 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.253666 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:18:40.255569 11748 heartbeater.cc:347] Connected to a master server at 127.27.18.254:41953
I20260808 18:18:40.255789 11748 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:40.256207 11748 heartbeater.cc:511] Master 127.27.18.254:41953 requested a full tablet report, sending...
I20260808 18:18:40.257612 11456 ts_manager.cc:194] Registered new tserver with Master: e91cdee575f34f8e9bc6e384ce4f2f7e (127.27.18.193:43713)
I20260808 18:18:40.258693 11456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:36999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:18:40.269719 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41953
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:40.376667 11754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:40.376889 11754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:40.376945 11754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:40.380864 11754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:40.380913 11754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:40.380964 11754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:18:40.385560 11754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:40.386426 11754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:40.387346 11754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:40.396366 11772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:40.396366 11765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:40.396418 11764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:40.396601 11754 server_base.cc:1034] running on GCE node
I20260808 18:18:40.396994 11754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:40.397552 11754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:40.398725 11754 hybrid_clock.cc:648] HybridClock initialized: now 1786213120398711 us; error 37 us; skew 500 ppm
Aug 08 18:18:40 dist-test-slave-x4qh krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213120, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:40.401271 11754 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:18:40.402089 11754 webserver.cc:460] Webserver started at http://127.27.18.194:43057/ using document root <none> and password file <none>
I20260808 18:18:40.402593 11754 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:40.402649 11754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:40.402866 11754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:40.404697 11754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "beeb281401eb4e339fce1e403f8fecdb"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "c4b49f31852df38694ed6829d317f9f8"
server_key_iv: "90ca0ecf4ebe25565ec58eb711e77f2d"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.405275 11754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "beeb281401eb4e339fce1e403f8fecdb"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "c4b49f31852df38694ed6829d317f9f8"
server_key_iv: "90ca0ecf4ebe25565ec58eb711e77f2d"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.408591 11754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:18:40.410784 11781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.411936 11754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:40.412079 11754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "beeb281401eb4e339fce1e403f8fecdb"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "c4b49f31852df38694ed6829d317f9f8"
server_key_iv: "90ca0ecf4ebe25565ec58eb711e77f2d"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.412201 11754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:40.441598 11754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:40.444221 11754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:40.444432 11754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:40.444950 11754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:40.445734 11754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:40.445802 11754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.445869 11754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:40.445916 11754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.454576 11754 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:34737
I20260808 18:18:40.454603 11943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:34737 every 8 connection(s)
I20260808 18:18:40.455441 11754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:18:40 dist-test-slave-x4qh krb5kdc[11385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213120, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:40.465629 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 11754
I20260808 18:18:40.465781 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:18:40.466078 27723 external_mini_cluster.cc:1428] Setting key ee9eb51baf07d9acbec74203f93dd3d2
I20260808 18:18:40.468473 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.457096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:56961 (local address 127.27.18.254:41953)
0808 18:18:40.457226 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.457229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.457986 (+ 757us) server_negotiation.cc:407] Connection header received
0808 18:18:40.459316 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.459320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.459393 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.459495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:40.460526 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.461046 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.462135 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.462373 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.462483 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.464782 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:40.464785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:40.464787 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:40.464818 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:40.466328 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.466860 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.466864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.466866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.466950 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.467275 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.467280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.467282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.467556 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:40.467660 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.468122 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.468258 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:18:40.469004 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.457428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41953 (local address 127.27.18.194:56961)
0808 18:18:40.457845 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.457883 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.459101 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.459658 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.459664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.459983 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:40.460386 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.460397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.461182 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.461185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.462006 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.462014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.462534 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.462556 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.462714 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.463172 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:40.463188 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:40.464629 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:40.466442 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:40.466448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:40.466479 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:40.466741 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:40.467045 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:40.467049 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:40.467051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:40.467177 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:40.467680 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:40.467685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:40.467887 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.468034 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.468249 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
I20260808 18:18:40.470182 11945 heartbeater.cc:347] Connected to a master server at 127.27.18.254:41953
I20260808 18:18:40.470415 11945 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:40.470846 11945 heartbeater.cc:511] Master 127.27.18.254:41953 requested a full tablet report, sending...
I20260808 18:18:40.471909 11456 ts_manager.cc:194] Registered new tserver with Master: beeb281401eb4e339fce1e403f8fecdb (127.27.18.194:34737)
I20260808 18:18:40.472749 11456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:56961
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:18:40.493779 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41953
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:18:40.598538 11952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:18:40.598735 11952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:18:40.598773 11952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:18:40.602475 11952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:18:40.602517 11952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:18:40.602568 11952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:18:40.607410 11952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:18:40.608242 11952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:18:40.609107 11952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:18:40.618049 11967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:40.618059 11965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:18:40.618096 11959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:18:40.618322 11952 server_base.cc:1034] running on GCE node
I20260808 18:18:40.618772 11952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:18:40.619341 11952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:18:40.620549 11952 hybrid_clock.cc:648] HybridClock initialized: now 1786213120620520 us; error 39 us; skew 500 ppm
Aug 08 18:18:40 dist-test-slave-x4qh krb5kdc[11385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213120, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:18:40.623006 11952 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:18:40.623842 11952 webserver.cc:460] Webserver started at http://127.27.18.195:40513/ using document root <none> and password file <none>
I20260808 18:18:40.624300 11952 fs_manager.cc:359] Metadata directory not provided
I20260808 18:18:40.624367 11952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:18:40.624518 11952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:18:40.626328 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "fc336bd455b14d4f879fe6be69e18a6c"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "bdf0daee447377ef11677ba29981a47f"
server_key_iv: "b40ec68da11db9ea6f0bd8aa3d1554bf"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.626818 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "fc336bd455b14d4f879fe6be69e18a6c"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "bdf0daee447377ef11677ba29981a47f"
server_key_iv: "b40ec68da11db9ea6f0bd8aa3d1554bf"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.630148 11952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:18:40.632339 11976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.633415 11952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:18:40.633558 11952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "fc336bd455b14d4f879fe6be69e18a6c"
format_stamp: "Formatted at 2026-08-08 18:18:40 on dist-test-slave-x4qh"
server_key: "bdf0daee447377ef11677ba29981a47f"
server_key_iv: "b40ec68da11db9ea6f0bd8aa3d1554bf"
server_key_version: "encryptionkey@0"
I20260808 18:18:40.633658 11952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:18:40.655507 11952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:18:40.658479 11952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:18:40.658670 11952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:18:40.659209 11952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:18:40.660014 11952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:18:40.660085 11952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.660153 11952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:18:40.660199 11952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:18:40.668795 11952 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:36533
I20260808 18:18:40.668833 12166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:36533 every 8 connection(s)
I20260808 18:18:40.669620 11952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
Aug 08 18:18:40 dist-test-slave-x4qh krb5kdc[11385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213120, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:18:40.679412 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 11952
I20260808 18:18:40.679553 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:18:40.679961 27723 external_mini_cluster.cc:1428] Setting key 97daf0c46e595dc53b4d5188b3ab8e55
I20260808 18:18:40.682399 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.671346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:42241 (local address 127.27.18.254:41953)
0808 18:18:40.671560 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.671565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.672389 (+ 824us) server_negotiation.cc:407] Connection header received
0808 18:18:40.673638 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.673641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.673685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.673757 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:40.674791 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.675295 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.676375 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.676578 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.676653 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.678825 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:40.678828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:40.678830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:40.678853 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:40.680362 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.680934 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.680937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.680938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.681023 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.681395 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.681397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.681398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.681551 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:40.681622 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.682179 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.682267 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 18:18:40.683116 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.671730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:41953 (local address 127.27.18.195:42241)
0808 18:18:40.672238 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.672279 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.673459 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.673916 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.673921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.674237 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:18:40.674669 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.674682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.675454 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.675457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.676263 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.676272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.676701 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.676703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.676830 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.677297 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:18:40.677320 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:18:40.678665 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:18:40.680489 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:40.680494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:40.680503 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:40.680832 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:40.681119 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:18:40.681125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:18:40.681129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:18:40.681307 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:18:40.681650 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:18:40.681655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:18:40.681939 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.682121 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.682340 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":133,"threads_started":1}
I20260808 18:18:40.684193 12167 heartbeater.cc:347] Connected to a master server at 127.27.18.254:41953
I20260808 18:18:40.684423 12167 heartbeater.cc:464] Registering TS with master...
I20260808 18:18:40.684818 12167 heartbeater.cc:511] Master 127.27.18.254:41953 requested a full tablet report, sending...
I20260808 18:18:40.685741 11456 ts_manager.cc:194] Registered new tserver with Master: fc336bd455b14d4f879fe6be69e18a6c (127.27.18.195:36533)
I20260808 18:18:40.686246 11456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:42241
I20260808 18:18:40.693328 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:18:40.701730 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.695030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.27.18.254:41953)
0808 18:18:40.695140 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.695144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.695325 (+ 181us) server_negotiation.cc:407] Connection header received
0808 18:18:40.695482 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.695485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.695521 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.695608 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:18:40.696243 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.696689 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.697450 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.697683 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.697784 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.698950 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:18:40.698954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:18:40.698956 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:18:40.698979 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:18:40.700123 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.700524 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.700527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.700529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.700607 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:18:40.700889 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:18:40.700893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:18:40.700894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:18:40.701104 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:18:40.701254 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.701481 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.701582 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:18:40.704406 11456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:18:40.707458 11456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:18:40.721513 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.715311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43713 (local address 127.0.0.1:51734)
0808 18:18:40.715940 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.716059 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.716197 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.716736 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.716744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.716769 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:40.716890 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.716898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.718268 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.718271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.719614 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.719627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.720968 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.720971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.721080 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.721116 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.721245 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.721373 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":39,"thread_start_us":106,"threads_started":1}
I20260808 18:18:40.721562 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.715486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.27.18.193:43713)
0808 18:18:40.715757 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.715774 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.716151 (+ 377us) server_negotiation.cc:407] Connection header received
0808 18:18:40.716375 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.716383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.716559 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.716724 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:40.717020 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.718119 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.719756 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.720677 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.720773 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.720908 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.721318 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.721425 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":105,"threads_started":1}
I20260808 18:18:40.721782 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.715465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:34737 (local address 127.0.0.1:54730)
0808 18:18:40.717068 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.717079 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.717181 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.717616 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.717619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.717637 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:40.717730 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.717736 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.719218 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.719223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.720378 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.720388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.721408 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.721411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.721501 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.721524 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.721578 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.721646 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1539,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260808 18:18:40.722374 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.716482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.27.18.194:34737)
0808 18:18:40.716816 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.716828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.717170 (+ 342us) server_negotiation.cc:407] Connection header received
0808 18:18:40.717284 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.717288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.717467 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.717862 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:40.718004 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.719089 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.720515 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.721270 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.721954 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.722118 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.722168 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.722252 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":111,"threads_started":1}
I20260808 18:18:40.722409 12185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.716075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:36533 (local address 127.0.0.1:41538)
0808 18:18:40.717359 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.717371 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.717446 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.717983 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.717986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.718003 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:40.718100 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.718106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.719042 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.719046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.720645 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.720652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.722037 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.722041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.722128 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.722151 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.722215 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.722278 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"mutex_wait_us":164,"thread_start_us":89,"threads_started":1}
I20260808 18:18:40.723760 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.716183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41538 (local address 127.27.18.195:36533)
0808 18:18:40.717657 (+ 1474us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.717666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.717682 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:18:40.717735 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.717737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.717859 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.718004 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:40.718224 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.718912 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.720748 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.721329 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.721429 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.721556 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.723521 (+ 1965us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.723615 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1388,"thread_start_us":157,"threads_started":1}
I20260808 18:18:40.724576 11643 tablet_service.cc:1467] Processing CreateTablet for tablet d74dbd36c5654c6dac24a43519cc88c3 (DEFAULT_TABLE table=test-table [id=29b2b71edd41401fb0ddd6f0309a88ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:40.724958 11850 tablet_service.cc:1467] Processing CreateTablet for tablet d74dbd36c5654c6dac24a43519cc88c3 (DEFAULT_TABLE table=test-table [id=29b2b71edd41401fb0ddd6f0309a88ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:40.725690 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d74dbd36c5654c6dac24a43519cc88c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:40.725878 11850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d74dbd36c5654c6dac24a43519cc88c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:40.726231 12051 tablet_service.cc:1467] Processing CreateTablet for tablet d74dbd36c5654c6dac24a43519cc88c3 (DEFAULT_TABLE table=test-table [id=29b2b71edd41401fb0ddd6f0309a88ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:18:40.727157 12051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d74dbd36c5654c6dac24a43519cc88c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:18:40.731328 12191 tablet_bootstrap.cc:492] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Bootstrap starting.
I20260808 18:18:40.732860 12193 tablet_bootstrap.cc:492] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e: Bootstrap starting.
I20260808 18:18:40.733376 12195 tablet_bootstrap.cc:492] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Bootstrap starting.
I20260808 18:18:40.734318 12191 tablet_bootstrap.cc:654] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:40.735183 12191 log.cc:824] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:40.735582 12195 tablet_bootstrap.cc:654] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:40.735908 12193 tablet_bootstrap.cc:654] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e: Neither blocks nor log segments found. Creating new log.
I20260808 18:18:40.736814 12195 log.cc:824] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:40.737124 12193 log.cc:824] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:18:40.737319 12191 tablet_bootstrap.cc:492] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: No bootstrap required, opened a new log
I20260808 18:18:40.737529 12191 ts_tablet_manager.cc:1397] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:18:40.738722 12195 tablet_bootstrap.cc:492] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: No bootstrap required, opened a new log
I20260808 18:18:40.738897 12195 ts_tablet_manager.cc:1397] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:18:40.740621 12193 tablet_bootstrap.cc:492] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e: No bootstrap required, opened a new log
I20260808 18:18:40.740860 12193 ts_tablet_manager.cc:1397] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 18:18:40.741228 12191 raft_consensus.cc:348] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.741489 12191 raft_consensus.cc:374] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:40.741546 12191 raft_consensus.cc:729] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beeb281401eb4e339fce1e403f8fecdb, State: Initialized, Role: FOLLOWER
I20260808 18:18:40.741979 12191 consensus_queue.cc:260] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.742502 12195 raft_consensus.cc:348] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.742659 11945 heartbeater.cc:503] Master 127.27.18.254:41953 was elected leader, sending a full tablet report...
I20260808 18:18:40.742726 12195 raft_consensus.cc:374] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:40.742784 12195 raft_consensus.cc:729] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc336bd455b14d4f879fe6be69e18a6c, State: Initialized, Role: FOLLOWER
I20260808 18:18:40.742976 12191 ts_tablet_manager.cc:1428] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:18:40.743152 12195 consensus_queue.cc:260] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.743782 12167 heartbeater.cc:503] Master 127.27.18.254:41953 was elected leader, sending a full tablet report...
I20260808 18:18:40.744045 12195 ts_tablet_manager.cc:1428] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260808 18:18:40.745410 12193 raft_consensus.cc:348] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.745687 12193 raft_consensus.cc:374] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:18:40.745754 12193 raft_consensus.cc:729] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91cdee575f34f8e9bc6e384ce4f2f7e, State: Initialized, Role: FOLLOWER
I20260808 18:18:40.746146 12193 consensus_queue.cc:260] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.747169 11748 heartbeater.cc:503] Master 127.27.18.254:41953 was elected leader, sending a full tablet report...
I20260808 18:18:40.747177 12193 ts_tablet_manager.cc:1428] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 18:18:40.920753 12168 tablet.cc:2369] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:40.924748 12202 raft_consensus.cc:482] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:18:40.924961 12202 raft_consensus.cc:504] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.926085 12202 leader_election.cc:288] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91cdee575f34f8e9bc6e384ce4f2f7e (127.27.18.193:43713), fc336bd455b14d4f879fe6be69e18a6c (127.27.18.195:36533)
I20260808 18:18:40.930852 12189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.926533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:43191 (local address 127.27.18.195:36533)
0808 18:18:40.926641 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.926651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.926663 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:18:40.926708 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.926710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.926749 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.926855 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:40.927158 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.927981 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.929692 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.930239 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.930356 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.930405 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.930667 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.930732 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:18:40.930852 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.926435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:35167 (local address 127.27.18.193:43713)
0808 18:18:40.926575 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.926581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.926824 (+ 243us) server_negotiation.cc:407] Connection header received
0808 18:18:40.926937 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.926943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.926989 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.927047 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:18:40.927307 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.928039 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.929692 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.930239 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.930356 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.930405 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.930667 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.930732 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:18:40.930874 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.926300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:36533 (local address 127.27.18.194:43191)
0808 18:18:40.926410 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.926425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.926517 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.926882 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.926886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.926906 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:40.927021 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.927045 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.928118 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.928121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.929568 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.929580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.930358 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.930361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.930547 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.930561 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.930636 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.930734 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":244}
I20260808 18:18:40.930882 12208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.926311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43713 (local address 127.27.18.194:35167)
0808 18:18:40.926717 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:40.926728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:18:40.926846 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:18:40.927081 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:18:40.927087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:18:40.927108 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:18:40.927203 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.927207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.928141 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.928143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.929568 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:18:40.929580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.930358 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.930361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:18:40.930508 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.930562 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 18:18:40.930636 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 18:18:40.930730 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":261,"thread_start_us":70,"threads_started":1}
I20260808 18:18:40.931670 11675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74dbd36c5654c6dac24a43519cc88c3" candidate_uuid: "beeb281401eb4e339fce1e403f8fecdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" is_pre_election: true
I20260808 18:18:40.931661 12082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74dbd36c5654c6dac24a43519cc88c3" candidate_uuid: "beeb281401eb4e339fce1e403f8fecdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc336bd455b14d4f879fe6be69e18a6c" is_pre_election: true
I20260808 18:18:40.931957 12082 raft_consensus.cc:2454] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beeb281401eb4e339fce1e403f8fecdb in term 0.
I20260808 18:18:40.931957 11675 raft_consensus.cc:2454] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beeb281401eb4e339fce1e403f8fecdb in term 0.
I20260808 18:18:40.932451 11786 leader_election.cc:302] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beeb281401eb4e339fce1e403f8fecdb, e91cdee575f34f8e9bc6e384ce4f2f7e; no voters:
I20260808 18:18:40.932734 12202 raft_consensus.cc:2781] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:18:40.932838 12202 raft_consensus.cc:482] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:18:40.932894 12202 raft_consensus.cc:3037] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:40.933874 12202 raft_consensus.cc:504] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.934185 12202 leader_election.cc:288] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [CANDIDATE]: Term 1 election: Requested vote from peers e91cdee575f34f8e9bc6e384ce4f2f7e (127.27.18.193:43713), fc336bd455b14d4f879fe6be69e18a6c (127.27.18.195:36533)
I20260808 18:18:40.934491 12082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74dbd36c5654c6dac24a43519cc88c3" candidate_uuid: "beeb281401eb4e339fce1e403f8fecdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc336bd455b14d4f879fe6be69e18a6c"
I20260808 18:18:40.934531 11675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d74dbd36c5654c6dac24a43519cc88c3" candidate_uuid: "beeb281401eb4e339fce1e403f8fecdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e"
I20260808 18:18:40.934623 12082 raft_consensus.cc:3037] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:40.934641 11675 raft_consensus.cc:3037] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:18:40.935672 11675 raft_consensus.cc:2454] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beeb281401eb4e339fce1e403f8fecdb in term 1.
I20260808 18:18:40.935740 12082 raft_consensus.cc:2454] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beeb281401eb4e339fce1e403f8fecdb in term 1.
I20260808 18:18:40.935972 11786 leader_election.cc:302] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beeb281401eb4e339fce1e403f8fecdb, e91cdee575f34f8e9bc6e384ce4f2f7e; no voters:
I20260808 18:18:40.936153 12202 raft_consensus.cc:2781] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:18:40.936383 12202 raft_consensus.cc:686] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 1 LEADER]: Becoming Leader. State: Replica: beeb281401eb4e339fce1e403f8fecdb, State: Running, Role: LEADER
I20260808 18:18:40.936694 12202 consensus_queue.cc:237] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } }
I20260808 18:18:40.939710 11454 catalog_manager.cc:5495] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb reported cstate change: term changed from 0 to 1, leader changed from <none> to beeb281401eb4e339fce1e403f8fecdb (127.27.18.194). New cstate: current_term: 1 leader_uuid: "beeb281401eb4e339fce1e403f8fecdb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beeb281401eb4e339fce1e403f8fecdb" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 } health_report { overall_health: UNKNOWN } } }
W20260808 18:18:40.956517 11948 tablet.cc:2369] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:18:40.962580 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:40.958951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.27.18.194:34737)
0808 18:18:40.959057 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:18:40.959062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:18:40.959171 (+ 109us) server_negotiation.cc:407] Connection header received
0808 18:18:40.959349 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:18:40.959351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:18:40.959439 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:18:40.959563 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:18:40.959813 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.960283 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.961148 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:18:40.961390 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:18:40.961503 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:18:40.961817 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:18:40.962171 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:18:40.962301 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:18:40.962373 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 18:18:40.962426 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:18:40.968269 12082 raft_consensus.cc:1261] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 1 FOLLOWER]: Refusing update from remote peer beeb281401eb4e339fce1e403f8fecdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:40.968405 11675 raft_consensus.cc:1261] T d74dbd36c5654c6dac24a43519cc88c3 P e91cdee575f34f8e9bc6e384ce4f2f7e [term 1 FOLLOWER]: Refusing update from remote peer beeb281401eb4e339fce1e403f8fecdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:18:40.969000 12202 consensus_queue.cc:1035] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc336bd455b14d4f879fe6be69e18a6c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 36533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:40.969197 12210 consensus_queue.cc:1035] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91cdee575f34f8e9bc6e384ce4f2f7e" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:18:40.977286 12215 mvcc.cc:205] Tried to move back new op lower bound from 7316328943479009280 to 7316328943359188992. Current Snapshot: MvccSnapshot[applied={T|T < 7316328943479009280}]
I20260808 18:18:40.977631 12213 mvcc.cc:205] Tried to move back new op lower bound from 7316328943479009280 to 7316328943359188992. Current Snapshot: MvccSnapshot[applied={T|T < 7316328943479009280}]
I20260808 18:18:40.978598 12216 mvcc.cc:205] Tried to move back new op lower bound from 7316328943479009280 to 7316328943359188992. Current Snapshot: MvccSnapshot[applied={T|T < 7316328943479009280}]
I20260808 18:18:40.987794 11454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:40.987924 11454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:18:40.990170 11454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5028adf0102d44c2898f0fc725045f08: Sending DeleteTablet for 3 replicas of tablet d74dbd36c5654c6dac24a43519cc88c3
I20260808 18:18:40.990880 11643 tablet_service.cc:1514] Processing DeleteTablet for tablet d74dbd36c5654c6dac24a43519cc88c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:40 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:51734
I20260808 18:18:40.990952 12051 tablet_service.cc:1514] Processing DeleteTablet for tablet d74dbd36c5654c6dac24a43519cc88c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:40 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:41538
I20260808 18:18:40.991124 11850 tablet_service.cc:1514] Processing DeleteTablet for tablet d74dbd36c5654c6dac24a43519cc88c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:18:40 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:54730
I20260808 18:18:40.991467 12228 tablet_replica.cc:330] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: stopping tablet replica
I20260808 18:18:40.991745 12228 raft_consensus.cc:2229] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:18:40.992027 12228 raft_consensus.cc:2258] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:40.992159 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 11548
I20260808 18:18:40.999082 12229 tablet_replica.cc:330] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: stopping tablet replica
I20260808 18:18:40.999408 12229 raft_consensus.cc:2229] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:18:40.999466 12228 ts_tablet_manager.cc:1905] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:40.999826 12229 raft_consensus.cc:2258] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:18:41.001850 12229 ts_tablet_manager.cc:1905] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:18:41.002426 12228 ts_tablet_manager.cc:1918] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:41.002524 12228 log.cc:1196] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/d74dbd36c5654c6dac24a43519cc88c3
I20260808 18:18:41.002866 12228 ts_tablet_manager.cc:1939] T d74dbd36c5654c6dac24a43519cc88c3 P fc336bd455b14d4f879fe6be69e18a6c: Deleting consensus metadata
I20260808 18:18:41.004071 11430 catalog_manager.cc:4841] TS fc336bd455b14d4f879fe6be69e18a6c (127.27.18.195:36533): tablet d74dbd36c5654c6dac24a43519cc88c3 (table test-table [id=29b2b71edd41401fb0ddd6f0309a88ea]) successfully deleted
I20260808 18:18:41.004560 12229 ts_tablet_manager.cc:1918] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:18:41.004674 12229 log.cc:1196] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/d74dbd36c5654c6dac24a43519cc88c3
I20260808 18:18:41.005038 12229 ts_tablet_manager.cc:1939] T d74dbd36c5654c6dac24a43519cc88c3 P beeb281401eb4e339fce1e403f8fecdb: Deleting consensus metadata
I20260808 18:18:41.005954 11430 catalog_manager.cc:4841] TS beeb281401eb4e339fce1e403f8fecdb (127.27.18.194:34737): tablet d74dbd36c5654c6dac24a43519cc88c3 (table test-table [id=29b2b71edd41401fb0ddd6f0309a88ea]) successfully deleted
W20260808 18:18:41.008677 11433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:43713 (error 108)
I20260808 18:18:41.009485 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 11754
I20260808 18:18:41.010246 12185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:18:41.009351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43713 (local address 127.0.0.1:51748)
0808 18:18:41.010091 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 18:18:41.010161 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:43713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645}
W20260808 18:18:41.015513 11433 catalog_manager.cc:4584] TS e91cdee575f34f8e9bc6e384ce4f2f7e (127.27.18.193:43713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d74dbd36c5654c6dac24a43519cc88c3: Network error: Client connection negotiation failed: client connection to 127.27.18.193:43713: connect: Connection refused (error 111)
I20260808 18:18:41.022863 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 11952
I20260808 18:18:41.035269 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 11408
2026-08-08T18:18:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2555 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:18:41 dist-test-slave-x4qh krb5kdc[12233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:18:41 dist-test-slave-x4qh krb5kdc[12233](info): set up 2 sockets
Aug 08 18:18:41 dist-test-slave-x4qh krb5kdc[12233](info): commencing operation
krb5kdc: starting...
W20260808 18:18:42.229176 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:18:42 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:18:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/rangeradmin_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/rangeradmin_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/rangerlookup_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/rangerlookup_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
I20260808 18:18:42.390012 27723 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPL43Mv/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/postgres -l logfile start
2026-08-08 18:18:44.590 UTC [12277] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 18:18:44.590 UTC [12277] LOG: listening on IPv4 address "127.27.18.212", port 44821
2026-08-08 18:18:44.593 UTC [12277] LOG: listening on Unix socket "/tmp/.s.PGSQL.44821"
2026-08-08 18:18:44.626 UTC [12280] LOG: database system was shut down at 2026-08-08 18:18:43 UTC
2026-08-08 18:18:44.630 UTC [12277] LOG: database system is ready to accept connections
I20260808 18:18:45.718601 27723 mini_postgres.cc:96] Postgres bound to 44821
2026-08-08 18:18:45.726 UTC [12291] FATAL: database "slave" does not exist
127.27.18.212:44821 - accepting connections
I20260808 18:18:45.727336 27723 mini_ranger.cc:161] Starting Ranger...
I20260808 18:18:45.742501 27723 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 18:18:46.141970 27723 mini_ranger.cc:87] Created ranger Postgres database
I20260808 18:18:46.142073 27723 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-admin
2026-08-08 18:18:46,458 [I] DB FLAVOR :POSTGRES
2026-08-08 18:18:46,458 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:18:46,458 [I] Checking connection..
2026-08-08 18:18:46,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 18:18:46,719 [I] Checking connection passed.
2026-08-08 18:18:46,719 [I] --------- Verifying version history table ---------
2026-08-08 18:18:46,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:18:46,970 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:18:46,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:18:47,214 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:18:47,214 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 18:18:47,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 18:18:47.456 UTC [12394] WARNING: there is no transaction in progress
2026-08-08 18:18:47,465 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 18:18:47,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:18:47,709 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 18:18:47,710 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:18:47,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 18:18:47,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 18:18:48,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-08 18:18:48,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:18:48,652 [I] Table x_portal_user does not exist in database ranger
2026-08-08 18:18:48,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:18:48,899 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 18:18:48,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:18:49,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:18:49,398 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 18:18:49,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 18:18:50.278 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.294 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.308 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.322 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.338 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.354 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.415 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.421 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.433 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.442 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.452 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.464 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.473 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.478 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.484 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:50.489 UTC [12556] WARNING: there is no transaction in progress
2026-08-08 18:18:51,033 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 18:18:51,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 18:18:51,268 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 18:18:51,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:18:51,530 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 18:18:51,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:18:51,757 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 18:18:51,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:18:51,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:18:52,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-08 18:18:52,449 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 18:18:52,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 18:18:52,702 [I] DB_PATCHES have already been applied
I20260808 18:18:52.708490 27723 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPL43Mv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 18:18:52.708571 27723 mini_ranger.cc:193] Using host: 127.27.18.212
I20260808 18:18:52.713145 27723 mini_ranger.cc:227] Ranger admin URL: http://127.27.18.212:48227
Aug 08, 2026 6:18:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:18:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:18:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 18:18:54.021188 27723 mini_ranger.cc:160] Time spent starting Ranger: real 8.294s user 0.004s sys 0.008s
Aug 08, 2026 6:19:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:19:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 6:19:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:19:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:19:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 18:19:14.936686 27723 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/rangerkms_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/rangerkms_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.18.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.18.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 18:19:15 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213155, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 18:19:15.021025 27723 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 18:19:15.031667 27723 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 18:19:15.781881 27723 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 18:19:15.782011 27723 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms
I20260808 18:19:15.782124 27723 mini_ranger_kms.cc:227] Using postgres at 127.27.18.212:44821
2026-08-08 18:19:16,560 [I] DB FLAVOR :POSTGRES
2026-08-08 18:19:16,560 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:19:16,560 [I] Checking connection
2026-08-08 18:19:16,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 18:19:16,893 [I] connection success
2026-08-08 18:19:16,893 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 18:19:16,894 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 18:19:16,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 18:19:17,134 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 18:19:17,134 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:19:17,134 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 18:19:17,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:44821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 18:19:17,386 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 18:19:17.825434 27723 mini_ranger_kms.cc:310] Created kms service
I20260808 18:19:17.989880 27723 mini_ranger_kms.cc:326] Created kudu user
I20260808 18:19:18.032426 27723 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 18:19:18 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213155, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:18.406055 27723 mini_ranger_kms.cc:384] Added ranger policy
I20260808 18:19:18.406224 27723 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 18:19:18.406251 27723 mini_ranger_kms.cc:240] Using host: 127.27.18.212
I20260808 18:19:18.411226 27723 mini_ranger_kms.cc:276] Ranger KMS PID: 12826
I20260808 18:19:18.411429 27723 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.18.212:42179
Aug 08, 2026 6:19:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:19:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:19:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:19:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:19:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:19:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 18:19:21 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213161, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.18.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:19:21.188956 27723 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.168s user 0.011s sys 0.654s
I20260808 18:19:21.189100 27723 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 18:19:21.189227 27723 mini_ranger_kms.cc:401] 127.27.18.212:42179/kms/v1/keys
Aug 08 18:19:21 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213161, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.18.212@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
Aug 08 18:19:21 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213161, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.18.212@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
Aug 08 18:19:24 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:19:24.283226 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44811
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:44811
--ranger_config_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:42179/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:19:24.434080 12894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:19:24.434331 12894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:19:24.434370 12894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:19:24.438431 12894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:19:24.438491 12894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:19:24.438509 12894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:19:24.438525 12894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:19:24.438541 12894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:19:24.444070 12894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.18.212:42179/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:44811
--ranger_config_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:19:24.445132 12894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:19:24.446177 12894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:19:24.455576 12905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:19:24.455753 12894 server_base.cc:1034] running on GCE node
W20260808 18:19:24.455657 12909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:24.455895 12903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:19:24.456467 12894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:19:24.457520 12894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:19:24.458725 12894 hybrid_clock.cc:648] HybridClock initialized: now 1786213164458714 us; error 29 us; skew 500 ppm
Aug 08 18:19:24 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213164, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:19:24.464833 12894 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:19:24.465906 12894 webserver.cc:460] Webserver started at http://127.27.18.254:40179/ using document root <none> and password file <none>
I20260808 18:19:24.466549 12894 fs_manager.cc:359] Metadata directory not provided
I20260808 18:19:24.466624 12894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:19:24.466847 12894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:19:24 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213164, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:24.514653 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "f26a3a65d1e44392a1cafe090c3366c6"
format_stamp: "Formatted at 2026-08-08 18:19:24 on dist-test-slave-x4qh"
server_key: "459840b74fd5e4a210cec57b0f11b2ce"
server_key_iv: "a3ecdcaff7c8683ec559b4a6497886c9"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:24.515331 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "f26a3a65d1e44392a1cafe090c3366c6"
format_stamp: "Formatted at 2026-08-08 18:19:24 on dist-test-slave-x4qh"
server_key: "459840b74fd5e4a210cec57b0f11b2ce"
server_key_iv: "a3ecdcaff7c8683ec559b4a6497886c9"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:24.519263 12894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 18:19:24.537001 12919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:24.538626 12894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 18:19:24.538774 12894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "f26a3a65d1e44392a1cafe090c3366c6"
format_stamp: "Formatted at 2026-08-08 18:19:24 on dist-test-slave-x4qh"
server_key: "459840b74fd5e4a210cec57b0f11b2ce"
server_key_iv: "a3ecdcaff7c8683ec559b4a6497886c9"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:24.538913 12894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:19:24.563834 12894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:19:24.567354 12894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:19:24.567678 12894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:19:24.585549 12894 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:44811
I20260808 18:19:24.585685 12989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:44811 every 8 connection(s)
I20260808 18:19:24.586918 12894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:19:24.589962 12990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:19:24.591495 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 12894
I20260808 18:19:24.591643 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:19:24.591972 27723 external_mini_cluster.cc:1428] Setting key 6fb26a9d65ffce883ae4ef51253b98e4
I20260808 18:19:24.597867 12990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6: Bootstrap starting.
I20260808 18:19:24.600934 12990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6: Neither blocks nor log segments found. Creating new log.
I20260808 18:19:24.602396 12990 log.cc:824] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:19:24 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:19:24.604944 12990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6: No bootstrap required, opened a new log
I20260808 18:19:24.609627 12990 raft_consensus.cc:348] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26a3a65d1e44392a1cafe090c3366c6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44811 } }
I20260808 18:19:24.609858 12990 raft_consensus.cc:374] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:19:24.609928 12990 raft_consensus.cc:729] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26a3a65d1e44392a1cafe090c3366c6, State: Initialized, Role: FOLLOWER
I20260808 18:19:24.610507 12990 consensus_queue.cc:260] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26a3a65d1e44392a1cafe090c3366c6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44811 } }
I20260808 18:19:24.610715 12990 raft_consensus.cc:388] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:19:24.610886 12990 raft_consensus.cc:482] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:19:24.611057 12990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:19:24.611937 12996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:24.593787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.27.18.254:44811)
0808 18:19:24.594452 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:24.594502 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:24.594553 (+ 51us) server_negotiation.cc:407] Connection header received
0808 18:19:24.595966 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:24.595992 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:24.596412 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:24.596813 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:19:24.597663 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:24.599373 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:24.600629 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:24.601455 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:24.601617 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:24.604762 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:19:24.604770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:19:24.604789 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:19:24.604821 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:19:24.608298 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:24.608961 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:24.608968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:24.608974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:24.609116 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:24.609449 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:24.609452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:24.609454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:24.609973 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:19:24.610243 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:24.610602 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 18:19:24.610888 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":211,"threads_started":1}
I20260808 18:19:24.612251 12990 raft_consensus.cc:504] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26a3a65d1e44392a1cafe090c3366c6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44811 } }
I20260808 18:19:24.612612 12990 leader_election.cc:302] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f26a3a65d1e44392a1cafe090c3366c6; no voters:
I20260808 18:19:24.612967 12990 leader_election.cc:288] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:19:24.613229 12998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:19:24.613673 12998 raft_consensus.cc:686] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [term 1 LEADER]: Becoming Leader. State: Replica: f26a3a65d1e44392a1cafe090c3366c6, State: Running, Role: LEADER
I20260808 18:19:24.614012 12998 consensus_queue.cc:237] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26a3a65d1e44392a1cafe090c3366c6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44811 } }
I20260808 18:19:24.614423 12990 sys_catalog.cc:564] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:19:24.615518 12998 sys_catalog.cc:455] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f26a3a65d1e44392a1cafe090c3366c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26a3a65d1e44392a1cafe090c3366c6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44811 } } }
I20260808 18:19:24.615759 12998 sys_catalog.cc:458] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:19:24.616201 12998 sys_catalog.cc:455] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f26a3a65d1e44392a1cafe090c3366c6. Latest consensus state: current_term: 1 leader_uuid: "f26a3a65d1e44392a1cafe090c3366c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f26a3a65d1e44392a1cafe090c3366c6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44811 } } }
I20260808 18:19:24.616277 12998 sys_catalog.cc:458] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:19:24.617125 13010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:19:24.621891 13010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:19:24.623874 12990 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:19:24.635502 13010 catalog_manager.cc:1347] Generated new cluster ID: 60d5bc9ece30406c94a890bd0f43fcb2
I20260808 18:19:24.635579 13010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:19:24.670037 13010 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:19:24.671918 13010 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:19:24.697366 13010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6: Generated new TSK 0
I20260808 18:19:24.698144 13010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:19:25.662554 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44811
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:19:25.842754 13055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:19:25.843022 13055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:19:25.843099 13055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:19:25.847357 13055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:19:25.847488 13055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:19:25.847580 13055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:19:25.854596 13055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.18.212:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:19:25.855846 13055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:19:25.857105 13055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:19:25.870123 13069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:25.870146 13071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:25.870639 13076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:19:25.870930 13055 server_base.cc:1034] running on GCE node
I20260808 18:19:25.871429 13055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:19:25.872423 13055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:19:25.873813 13055 hybrid_clock.cc:648] HybridClock initialized: now 1786213165873776 us; error 61 us; skew 500 ppm
Aug 08 18:19:25 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213165, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:19:25.878639 13055 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:19:25.879964 13055 webserver.cc:460] Webserver started at http://127.27.18.193:36577/ using document root <none> and password file <none>
I20260808 18:19:25.880749 13055 fs_manager.cc:359] Metadata directory not provided
I20260808 18:19:25.880910 13055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:19:25.881215 13055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:19:25 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213165, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:25.902278 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "7e2ba73bbdd0403ea363925f8f52d8f7"
format_stamp: "Formatted at 2026-08-08 18:19:25 on dist-test-slave-x4qh"
server_key: "7c46df88087061fb066dd0744cca1727"
server_key_iv: "20cb537834e020310af3802b4db84d55"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:25.902978 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "7e2ba73bbdd0403ea363925f8f52d8f7"
format_stamp: "Formatted at 2026-08-08 18:19:25 on dist-test-slave-x4qh"
server_key: "7c46df88087061fb066dd0744cca1727"
server_key_iv: "20cb537834e020310af3802b4db84d55"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:25.906971 13055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 18:19:25.923123 13086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:25.924750 13055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:19:25.924885 13055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "7e2ba73bbdd0403ea363925f8f52d8f7"
format_stamp: "Formatted at 2026-08-08 18:19:25 on dist-test-slave-x4qh"
server_key: "7c46df88087061fb066dd0744cca1727"
server_key_iv: "20cb537834e020310af3802b4db84d55"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:25.925016 13055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:19:25.951615 13055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:19:25.962457 13055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:19:25.962802 13055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:19:25.963690 13055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:19:25.964946 13055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:19:25.965109 13055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:25.965250 13055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:19:25.965337 13055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:25.998472 13055 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:33141
I20260808 18:19:25.999784 13055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:19:26.002832 13247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:33141 every 8 connection(s)
I20260808 18:19:26.023988 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 13055
I20260808 18:19:26.024188 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:19:26.024756 27723 external_mini_cluster.cc:1428] Setting key 566cf5a2225a4bd12c47fa5e66e03d0d
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213165, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:19:26.050065 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.004052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:42549 (local address 127.27.18.254:44811)
0808 18:19:26.004566 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:26.004573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:26.025509 (+ 20936us) server_negotiation.cc:407] Connection header received
0808 18:19:26.033081 (+ 7572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:26.033085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:26.033146 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:26.033206 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:19:26.035526 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.036437 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.038493 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.038944 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.039069 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.043357 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:19:26.043361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:19:26.043363 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:19:26.043424 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:19:26.045720 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.048226 (+ 2506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.048232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.048234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.048352 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.048811 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.048816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.048819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.049057 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:19:26.049149 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:26.049765 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 18:19:26.049887 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":259,"threads_started":1}
I20260808 18:19:26.051057 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.004463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44811 (local address 127.27.18.193:42549)
0808 18:19:26.005220 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:26.005284 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:26.032787 (+ 27503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:26.033703 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:26.033713 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:26.034436 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:19:26.035327 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:26.035351 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.036613 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.036617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:26.038335 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:26.038350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.039434 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.039439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:26.039797 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.040668 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:19:26.040706 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:19:26.043154 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:19:26.045929 (+ 2775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:19:26.045938 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:19:26.045966 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:19:26.046394 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:19:26.048490 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:19:26.048494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:19:26.048497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:19:26.048694 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:19:26.049222 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:19:26.049230 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:19:26.049521 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 18:19:26.049710 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 18:19:26.050009 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":265,"threads_started":1}
I20260808 18:19:26.052966 13248 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44811
I20260808 18:19:26.053223 13248 heartbeater.cc:464] Registering TS with master...
I20260808 18:19:26.053704 13248 heartbeater.cc:511] Master 127.27.18.254:44811 requested a full tablet report, sending...
I20260808 18:19:26.055584 12941 ts_manager.cc:194] Registered new tserver with Master: 7e2ba73bbdd0403ea363925f8f52d8f7 (127.27.18.193:33141)
I20260808 18:19:26.057618 12941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:42549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:19:26.068265 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44811
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 18:19:26.267980 13257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:19:26.268368 13257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:19:26.268479 13257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:19:26.275568 13257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:19:26.275671 13257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:19:26.275790 13257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:19:26.284617 13257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.18.212:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:19:26.286252 13257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:19:26.287794 13257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:19:26.302340 13266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:26.302289 13264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:26.302934 13268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:19:26.302968 13257 server_base.cc:1034] running on GCE node
I20260808 18:19:26.303681 13257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:19:26.304373 13257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:19:26.305549 13257 hybrid_clock.cc:648] HybridClock initialized: now 1786213166305476 us; error 94 us; skew 500 ppm
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:19:26.309108 13257 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:19:26.310356 13257 webserver.cc:460] Webserver started at http://127.27.18.194:36411/ using document root <none> and password file <none>
I20260808 18:19:26.311288 13257 fs_manager.cc:359] Metadata directory not provided
I20260808 18:19:26.311445 13257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:19:26.311867 13257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:26.330137 13257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "f781cd25f6c54b25a7937200b6f1b7d1"
format_stamp: "Formatted at 2026-08-08 18:19:26 on dist-test-slave-x4qh"
server_key: "3653d741a89802aae5b8b575a10aae64"
server_key_iv: "bace1f478691a639e9fcdcb71cf22524"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:26.331084 13257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "f781cd25f6c54b25a7937200b6f1b7d1"
format_stamp: "Formatted at 2026-08-08 18:19:26 on dist-test-slave-x4qh"
server_key: "3653d741a89802aae5b8b575a10aae64"
server_key_iv: "bace1f478691a639e9fcdcb71cf22524"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:26.337296 13257 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.003s
I20260808 18:19:26.363925 13278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:26.365756 13257 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.000s
I20260808 18:19:26.365962 13257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "f781cd25f6c54b25a7937200b6f1b7d1"
format_stamp: "Formatted at 2026-08-08 18:19:26 on dist-test-slave-x4qh"
server_key: "3653d741a89802aae5b8b575a10aae64"
server_key_iv: "bace1f478691a639e9fcdcb71cf22524"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:26.366202 13257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:19:26.390168 13257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:19:26.393894 13257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:19:26.394191 13257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:19:26.394974 13257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:19:26.396443 13257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:19:26.396572 13257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 18:19:26.396698 13257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:19:26.396782 13257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:26.426656 13257 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:39147
I20260808 18:19:26.427949 13257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:19:26.433099 13462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:39147 every 8 connection(s)
I20260808 18:19:26.436515 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 13257
I20260808 18:19:26.436662 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:19:26.436985 27723 external_mini_cluster.cc:1428] Setting key 1c79fd6b82b22880cf929f5f8b20844e
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:19:26.448056 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.432762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:45143 (local address 127.27.18.254:44811)
0808 18:19:26.432985 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:26.432993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:26.433884 (+ 891us) server_negotiation.cc:407] Connection header received
0808 18:19:26.435499 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:26.435503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:26.435558 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:26.435613 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:19:26.437104 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.437908 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.439862 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.440237 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.440309 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.443717 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:19:26.443721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:19:26.443724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:19:26.443750 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:19:26.445548 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.446203 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.446207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.446210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.446383 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.446815 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.446818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.446820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.447065 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:19:26.447175 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:26.447777 (+ 602us) server_negotiation.cc:299] Negotiation successful
0808 18:19:26.447903 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 18:19:26.449744 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.431572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44811 (local address 127.27.18.194:45143)
0808 18:19:26.433549 (+ 1977us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:26.433600 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:26.435224 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:26.436007 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:26.436015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:26.436354 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:19:26.436946 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:26.436963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.438284 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.438287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:26.439726 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:26.439737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.440500 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.440503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:26.440689 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.441384 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:19:26.441403 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:19:26.443517 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:19:26.445690 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:19:26.445695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:19:26.445709 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:19:26.446025 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:19:26.446514 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:19:26.446517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:19:26.446519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:19:26.446702 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:19:26.447200 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:19:26.447207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:19:26.447510 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 18:19:26.448468 (+ 958us) client_negotiation.cc:240] Negotiation successful
0808 18:19:26.448762 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1734,"thread_start_us":217,"threads_started":1}
I20260808 18:19:26.451025 13463 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44811
I20260808 18:19:26.451251 13463 heartbeater.cc:464] Registering TS with master...
I20260808 18:19:26.451911 13463 heartbeater.cc:511] Master 127.27.18.254:44811 requested a full tablet report, sending...
I20260808 18:19:26.453035 12941 ts_manager.cc:194] Registered new tserver with Master: f781cd25f6c54b25a7937200b6f1b7d1 (127.27.18.194:39147)
I20260808 18:19:26.453845 12941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:45143
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:19:26.485733 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:42179/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44811
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:19:26.652392 13469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:19:26.652675 13469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:19:26.652751 13469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:19:26.657253 13469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:19:26.657334 13469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:19:26.657464 13469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:19:26.664940 13469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.18.212:42179/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:19:26.666585 13469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:19:26.668217 13469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:19:26.681867 13479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:26.683073 13480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:19:26.683560 13485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:19:26.684882 13469 server_base.cc:1034] running on GCE node
I20260808 18:19:26.685309 13469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:19:26.686000 13469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:19:26.689296 13469 hybrid_clock.cc:648] HybridClock initialized: now 1786213166689265 us; error 40 us; skew 500 ppm
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:19:26.693199 13469 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:19:26.694268 13469 webserver.cc:460] Webserver started at http://127.27.18.195:45415/ using document root <none> and password file <none>
I20260808 18:19:26.694864 13469 fs_manager.cc:359] Metadata directory not provided
I20260808 18:19:26.694963 13469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:19:26.695175 13469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:26.712145 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "bf3cf883769a4bb481dff7f963a2f5f0"
format_stamp: "Formatted at 2026-08-08 18:19:26 on dist-test-slave-x4qh"
server_key: "b5ee5e70fe16c3c1d0de722480da2287"
server_key_iv: "3b428f3e3160ac280a17fef6263f1c14"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:26.713032 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "bf3cf883769a4bb481dff7f963a2f5f0"
format_stamp: "Formatted at 2026-08-08 18:19:26 on dist-test-slave-x4qh"
server_key: "b5ee5e70fe16c3c1d0de722480da2287"
server_key_iv: "3b428f3e3160ac280a17fef6263f1c14"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:26.729060 13469 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.003s sys 0.001s
I20260808 18:19:26.745203 13495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:26.747115 13469 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:19:26.747306 13469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "bf3cf883769a4bb481dff7f963a2f5f0"
format_stamp: "Formatted at 2026-08-08 18:19:26 on dist-test-slave-x4qh"
server_key: "b5ee5e70fe16c3c1d0de722480da2287"
server_key_iv: "3b428f3e3160ac280a17fef6263f1c14"
server_key_version: "kuduclusterkey@0"
I20260808 18:19:26.747550 13469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:19:26.795562 13469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:19:26.805718 13469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:19:26.806044 13469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:19:26.807114 13469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:19:26.808506 13469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:19:26.808629 13469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:19:26.808741 13469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:19:26.808821 13469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:26.912645 13469 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46391
I20260808 18:19:26.924095 13672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46391 every 8 connection(s)
I20260808 18:19:26.926039 13469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:19:26.935758 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 13469
I20260808 18:19:26.935917 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:19:26.936405 27723 external_mini_cluster.cc:1428] Setting key 9fc4745ad43ce9ebfaf4580eaaf008ad
Aug 08 18:19:26 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:19:26.963615 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.938946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:52387 (local address 127.27.18.254:44811)
0808 18:19:26.939915 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:26.939920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:26.947410 (+ 7490us) server_negotiation.cc:407] Connection header received
0808 18:19:26.947460 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:26.947463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:26.947511 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:26.947616 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:19:26.949556 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.950354 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.955108 (+ 4754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.955509 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.955593 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.959217 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:19:26.959221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:19:26.959224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:19:26.959251 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:19:26.961109 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.961780 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.961785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.961787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.961876 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.962310 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.962313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.962315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.962535 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:19:26.962610 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:26.963308 (+ 698us) server_negotiation.cc:299] Negotiation successful
0808 18:19:26.963456 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877}
I20260808 18:19:26.965446 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.938852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44811 (local address 127.27.18.195:52387)
0808 18:19:26.945744 (+ 6892us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:26.945802 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:26.947171 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:26.947923 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:26.947932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:26.948519 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:19:26.949376 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:26.949416 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.950505 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.950508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:26.954963 (+ 4455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:26.954975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.955853 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.955857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:26.956136 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.957023 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:19:26.957043 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:19:26.959016 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:19:26.961266 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:19:26.961273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:19:26.961288 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:19:26.961666 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:19:26.961994 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:19:26.961999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:19:26.962001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:19:26.962194 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:19:26.962694 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:19:26.962700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:19:26.963010 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 18:19:26.964001 (+ 991us) client_negotiation.cc:240] Negotiation successful
0808 18:19:26.964336 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6651,"thread_start_us":130,"threads_started":1}
I20260808 18:19:26.966923 13673 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44811
I20260808 18:19:26.967171 13673 heartbeater.cc:464] Registering TS with master...
I20260808 18:19:26.968389 13673 heartbeater.cc:511] Master 127.27.18.254:44811 requested a full tablet report, sending...
I20260808 18:19:26.969515 12941 ts_manager.cc:194] Registered new tserver with Master: bf3cf883769a4bb481dff7f963a2f5f0 (127.27.18.195:46391)
I20260808 18:19:26.970237 12941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:52387
I20260808 18:19:26.975031 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:19:26.995677 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:26.979617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.27.18.254:44811)
0808 18:19:26.979738 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:26.979743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:26.980843 (+ 1100us) server_negotiation.cc:407] Connection header received
0808 18:19:26.981025 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:26.981028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:26.981070 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:26.981117 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:19:26.982147 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.982954 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.987490 (+ 4536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:26.987822 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:26.987901 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:26.992207 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:19:26.992211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:19:26.992214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:19:26.992240 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:19:26.993990 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.994435 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.994439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.994440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.994511 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:19:26.994790 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:19:26.994793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:19:26.994795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:19:26.994978 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:19:26.995068 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:26.995403 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 18:19:26.995519 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:19:27.000416 12941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:19:27.005158 12941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:19:27.061043 13248 heartbeater.cc:503] Master 127.27.18.254:44811 was elected leader, sending a full tablet report...
I20260808 18:19:27.061937 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.038652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46391 (local address 127.0.0.1:60878)
0808 18:19:27.041751 (+ 3099us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.041798 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:27.042104 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:27.053948 (+ 11844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:27.053952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:27.053968 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:19:27.054150 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.054158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.055838 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.055841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.057403 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.057415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.061355 (+ 3940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.061359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.061497 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.061543 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 18:19:27.061653 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 18:19:27.061781 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2911,"mutex_wait_us":60,"thread_start_us":138,"threads_started":1}
I20260808 18:19:27.062418 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.039474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.27.18.195:46391)
0808 18:19:27.043607 (+ 4133us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:27.043618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:27.043645 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:19:27.043721 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:27.043726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:27.043915 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:27.044129 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:19:27.054688 (+ 10559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.055734 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.057750 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.058553 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.058663 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.058811 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:27.062147 (+ 3336us) server_negotiation.cc:299] Negotiation successful
0808 18:19:27.062268 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4014,"thread_start_us":83,"threads_started":1}
I20260808 18:19:27.063009 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.038668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:39147 (local address 127.0.0.1:33292)
0808 18:19:27.051718 (+ 13050us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.051732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:27.052067 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:27.052950 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:27.052956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:27.052979 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:19:27.053141 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.053150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.055874 (+ 2724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.055878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.057570 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.057582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.062615 (+ 5033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.062619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.062722 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.062747 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:19:27.062828 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:19:27.062889 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12964,"mutex_wait_us":227,"thread_start_us":1929,"threads_started":1}
I20260808 18:19:27.065688 13564 tablet_service.cc:1467] Processing CreateTablet for tablet ad5e4981cb7a47f2b4bf621437f7e96b (DEFAULT_TABLE table=test-table [id=86d0684e007742288a2bd55343ae8f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:19:27.066310 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.039821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.27.18.194:39147)
0808 18:19:27.051548 (+ 11727us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:27.051555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:27.052174 (+ 619us) server_negotiation.cc:407] Connection header received
0808 18:19:27.052360 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:27.052366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:27.052565 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:27.052770 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:19:27.053283 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.054525 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.058874 (+ 4349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.059705 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.059810 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.059941 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:27.066061 (+ 6120us) server_negotiation.cc:299] Negotiation successful
0808 18:19:27.066165 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11626,"thread_start_us":187,"threads_started":1}
I20260808 18:19:27.067899 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.041076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33141 (local address 127.0.0.1:40322)
0808 18:19:27.060093 (+ 19017us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.060109 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:27.060443 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:27.063219 (+ 2776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:27.063222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:27.063241 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:19:27.063345 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.063351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.064797 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.064800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.065906 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.065917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.067540 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.067544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.067639 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.067662 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:19:27.067730 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:19:27.067788 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18952,"thread_start_us":314,"threads_started":1}
I20260808 18:19:27.068522 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.041251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.27.18.193:33141)
0808 18:19:27.056306 (+ 15055us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:27.056313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:27.060554 (+ 4241us) server_negotiation.cc:407] Connection header received
0808 18:19:27.060646 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:27.060651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:27.060845 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:27.061191 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:19:27.063505 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.064654 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.066513 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.067239 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.067336 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.067477 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:27.068302 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 18:19:27.068387 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14946,"thread_start_us":123,"threads_started":1}
I20260808 18:19:27.069957 13352 tablet_service.cc:1467] Processing CreateTablet for tablet ad5e4981cb7a47f2b4bf621437f7e96b (DEFAULT_TABLE table=test-table [id=86d0684e007742288a2bd55343ae8f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:19:27.071053 13352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5e4981cb7a47f2b4bf621437f7e96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:19:27.070958 13147 tablet_service.cc:1467] Processing CreateTablet for tablet ad5e4981cb7a47f2b4bf621437f7e96b (DEFAULT_TABLE table=test-table [id=86d0684e007742288a2bd55343ae8f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:19:27.071924 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5e4981cb7a47f2b4bf621437f7e96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:19:27.072284 13564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5e4981cb7a47f2b4bf621437f7e96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:19:27.091711 13702 tablet_bootstrap.cc:492] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: Bootstrap starting.
I20260808 18:19:27.094548 13702 tablet_bootstrap.cc:654] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: Neither blocks nor log segments found. Creating new log.
I20260808 18:19:27.095340 13702 log.cc:824] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:19:27.098932 13702 tablet_bootstrap.cc:492] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: No bootstrap required, opened a new log
I20260808 18:19:27.099224 13702 ts_tablet_manager.cc:1397] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 18:19:27.105197 13702 raft_consensus.cc:348] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.105576 13702 raft_consensus.cc:374] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:19:27.105679 13702 raft_consensus.cc:729] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2ba73bbdd0403ea363925f8f52d8f7, State: Initialized, Role: FOLLOWER
I20260808 18:19:27.106194 13702 consensus_queue.cc:260] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.107462 13702 ts_tablet_manager.cc:1428] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20260808 18:19:27.111760 13703 tablet_bootstrap.cc:492] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: Bootstrap starting.
I20260808 18:19:27.114596 13704 tablet_bootstrap.cc:492] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Bootstrap starting.
I20260808 18:19:27.114671 13703 tablet_bootstrap.cc:654] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: Neither blocks nor log segments found. Creating new log.
I20260808 18:19:27.116298 13703 log.cc:824] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: Log is configured to *not* fsync() on all Append() calls
I20260808 18:19:27.117945 13704 tablet_bootstrap.cc:654] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Neither blocks nor log segments found. Creating new log.
I20260808 18:19:27.118444 13703 tablet_bootstrap.cc:492] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: No bootstrap required, opened a new log
I20260808 18:19:27.118613 13703 ts_tablet_manager.cc:1397] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260808 18:19:27.125036 13703 raft_consensus.cc:348] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.125342 13703 raft_consensus.cc:374] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:19:27.125432 13703 raft_consensus.cc:729] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f781cd25f6c54b25a7937200b6f1b7d1, State: Initialized, Role: FOLLOWER
I20260808 18:19:27.125926 13703 consensus_queue.cc:260] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.127260 13703 ts_tablet_manager.cc:1428] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20260808 18:19:27.127684 13463 heartbeater.cc:503] Master 127.27.18.254:44811 was elected leader, sending a full tablet report...
I20260808 18:19:27.130800 13704 log.cc:824] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Log is configured to *not* fsync() on all Append() calls
I20260808 18:19:27.134164 13704 tablet_bootstrap.cc:492] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: No bootstrap required, opened a new log
I20260808 18:19:27.134404 13704 ts_tablet_manager.cc:1397] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.003s
I20260808 18:19:27.139545 13704 raft_consensus.cc:348] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.139925 13704 raft_consensus.cc:374] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:19:27.140015 13704 raft_consensus.cc:729] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3cf883769a4bb481dff7f963a2f5f0, State: Initialized, Role: FOLLOWER
I20260808 18:19:27.140513 13704 consensus_queue.cc:260] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.141566 13704 ts_tablet_manager.cc:1428] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:19:27.141940 13673 heartbeater.cc:503] Master 127.27.18.254:44811 was elected leader, sending a full tablet report...
W20260808 18:19:27.179952 13464 tablet.cc:2369] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:19:27.181035 13675 tablet.cc:2369] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:19:27.255311 13249 tablet.cc:2369] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 18:19:27 dist-test-slave-x4qh krb5kdc[12233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213166, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:19:27.318199 13714 raft_consensus.cc:482] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:19:27.318459 13714 raft_consensus.cc:504] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.319968 13714 leader_election.cc:288] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f781cd25f6c54b25a7937200b6f1b7d1 (127.27.18.194:39147), 7e2ba73bbdd0403ea363925f8f52d8f7 (127.27.18.193:33141)
I20260808 18:19:27.326783 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.320402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:39147 (local address 127.27.18.195:40133)
0808 18:19:27.321210 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.321224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:27.321315 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:27.322596 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:27.322599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:27.322624 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:19:27.322790 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.322799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.323966 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.323969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.325081 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.325093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.326276 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.326280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.326431 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.326458 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:19:27.326548 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 18:19:27.326635 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716}
I20260808 18:19:27.328208 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.321698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:40133 (local address 127.27.18.194:39147)
0808 18:19:27.321981 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:27.321987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:27.322005 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:19:27.322063 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:27.322066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:27.322117 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:27.322205 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:19:27.322947 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.323662 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.325400 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.326003 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.326127 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.326209 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:27.328005 (+ 1796us) server_negotiation.cc:299] Negotiation successful
0808 18:19:27.328084 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260808 18:19:27.329008 13380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5e4981cb7a47f2b4bf621437f7e96b" candidate_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" is_pre_election: true
I20260808 18:19:27.329344 13380 raft_consensus.cc:2454] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf3cf883769a4bb481dff7f963a2f5f0 in term 0.
I20260808 18:19:27.330209 13496 leader_election.cc:302] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf3cf883769a4bb481dff7f963a2f5f0, f781cd25f6c54b25a7937200b6f1b7d1; no voters:
I20260808 18:19:27.333887 13714 raft_consensus.cc:2781] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:19:27.333973 13714 raft_consensus.cc:482] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:19:27.334014 13714 raft_consensus.cc:3037] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:19:27.339394 13714 raft_consensus.cc:504] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.340279 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.321845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:58903 (local address 127.27.18.193:33141)
0808 18:19:27.322410 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:27.322415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:27.327114 (+ 4699us) server_negotiation.cc:407] Connection header received
0808 18:19:27.327187 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:27.327190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:27.327246 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:27.327313 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:19:27.335776 (+ 8463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.336421 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.337642 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.338292 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.338405 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.338457 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:27.340072 (+ 1615us) server_negotiation.cc:299] Negotiation successful
0808 18:19:27.340154 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477}
I20260808 18:19:27.341226 13380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5e4981cb7a47f2b4bf621437f7e96b" candidate_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f781cd25f6c54b25a7937200b6f1b7d1"
I20260808 18:19:27.341353 13380 raft_consensus.cc:3037] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:19:27.343688 13380 raft_consensus.cc:2454] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf3cf883769a4bb481dff7f963a2f5f0 in term 1.
I20260808 18:19:27.344074 13496 leader_election.cc:302] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf3cf883769a4bb481dff7f963a2f5f0, f781cd25f6c54b25a7937200b6f1b7d1; no voters:
I20260808 18:19:27.339994 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.320653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33141 (local address 127.27.18.195:58903)
0808 18:19:27.326943 (+ 6290us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.326958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:27.327028 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:27.335535 (+ 8507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:19:27.335540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:19:27.335564 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:19:27.335640 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.335650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.336550 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.336554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.337497 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:19:27.337510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.339595 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.339599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:19:27.339684 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.339706 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:19:27.339776 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:19:27.339849 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6228,"thread_start_us":263,"threads_started":1}
I20260808 18:19:27.345002 13714 leader_election.cc:288] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [CANDIDATE]: Term 1 election: Requested vote from peers f781cd25f6c54b25a7937200b6f1b7d1 (127.27.18.194:39147), 7e2ba73bbdd0403ea363925f8f52d8f7 (127.27.18.193:33141)
I20260808 18:19:27.345292 13176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5e4981cb7a47f2b4bf621437f7e96b" candidate_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" is_pre_election: true
I20260808 18:19:27.345634 13176 raft_consensus.cc:2454] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf3cf883769a4bb481dff7f963a2f5f0 in term 0.
I20260808 18:19:27.346235 13175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5e4981cb7a47f2b4bf621437f7e96b" candidate_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7"
I20260808 18:19:27.346342 13175 raft_consensus.cc:3037] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:19:27.345448 13714 raft_consensus.cc:2781] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:19:27.346632 13714 raft_consensus.cc:686] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 1 LEADER]: Becoming Leader. State: Replica: bf3cf883769a4bb481dff7f963a2f5f0, State: Running, Role: LEADER
I20260808 18:19:27.346957 13714 consensus_queue.cc:237] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } }
I20260808 18:19:27.348009 13175 raft_consensus.cc:2454] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf3cf883769a4bb481dff7f963a2f5f0 in term 1.
I20260808 18:19:27.350332 12940 catalog_manager.cc:5495] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf3cf883769a4bb481dff7f963a2f5f0 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf3cf883769a4bb481dff7f963a2f5f0" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46391 } health_report { overall_health: HEALTHY } } }
I20260808 18:19:27.425554 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.420133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60890 (local address 127.27.18.195:46391)
0808 18:19:27.420298 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 18:19:27.420304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:19:27.420511 (+ 207us) server_negotiation.cc:407] Connection header received
0808 18:19:27.420612 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:19:27.420615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:19:27.420689 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:19:27.420810 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:19:27.421236 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.422054 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.423281 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:19:27.423619 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:19:27.423723 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:19:27.424154 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:19:27.424755 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:19:27.424882 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 18:19:27.425316 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 18:19:27.425376 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:19:27.433182 13175 raft_consensus.cc:1261] T ad5e4981cb7a47f2b4bf621437f7e96b P 7e2ba73bbdd0403ea363925f8f52d8f7 [term 1 FOLLOWER]: Refusing update from remote peer bf3cf883769a4bb481dff7f963a2f5f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:19:27.434051 13717 consensus_queue.cc:1035] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2ba73bbdd0403ea363925f8f52d8f7" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:19:27.441023 13380 raft_consensus.cc:1261] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 1 FOLLOWER]: Refusing update from remote peer bf3cf883769a4bb481dff7f963a2f5f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:19:27.441717 13717 consensus_queue.cc:1035] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f781cd25f6c54b25a7937200b6f1b7d1" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:19:27.454967 13723 mvcc.cc:205] Tried to move back new op lower bound from 7316329133796696064 to 7316329133456637952. Current Snapshot: MvccSnapshot[applied={T|T < 7316329133796696064}]
I20260808 18:19:27.455274 13722 mvcc.cc:205] Tried to move back new op lower bound from 7316329133796696064 to 7316329133456637952. Current Snapshot: MvccSnapshot[applied={T|T < 7316329133796696064}]
I20260808 18:19:27.459205 13725 mvcc.cc:205] Tried to move back new op lower bound from 7316329133796696064 to 7316329133456637952. Current Snapshot: MvccSnapshot[applied={T|T < 7316329133796696064}]
I20260808 18:19:27.471994 12941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:19:27.472188 12941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:19:27.474858 12941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f26a3a65d1e44392a1cafe090c3366c6: Sending DeleteTablet for 3 replicas of tablet ad5e4981cb7a47f2b4bf621437f7e96b
I20260808 18:19:27.476634 13352 tablet_service.cc:1514] Processing DeleteTablet for tablet ad5e4981cb7a47f2b4bf621437f7e96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:19:27 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33292
I20260808 18:19:27.477294 13147 tablet_service.cc:1514] Processing DeleteTablet for tablet ad5e4981cb7a47f2b4bf621437f7e96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:19:27 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:40322
I20260808 18:19:27.478292 13564 tablet_service.cc:1514] Processing DeleteTablet for tablet ad5e4981cb7a47f2b4bf621437f7e96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:19:27 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:60878
I20260808 18:19:27.479974 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 13055
I20260808 18:19:27.484515 13739 tablet_replica.cc:330] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1: stopping tablet replica
I20260808 18:19:27.484916 13739 raft_consensus.cc:2229] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:19:27.485221 13739 raft_consensus.cc:2258] T ad5e4981cb7a47f2b4bf621437f7e96b P f781cd25f6c54b25a7937200b6f1b7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:19:27.487581 13744 tablet_replica.cc:330] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: stopping tablet replica
I20260808 18:19:27.491730 13744 raft_consensus.cc:2229] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:19:27.492309 13744 raft_consensus.cc:2258] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 18:19:27.495862 12920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:33141 (error 108)
I20260808 18:19:27.497028 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 13257
I20260808 18:19:27.504191 13744 ts_tablet_manager.cc:1905] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:19:27.516036 13744 ts_tablet_manager.cc:1918] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:19:27.516235 13744 log.cc:1196] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/ad5e4981cb7a47f2b4bf621437f7e96b
I20260808 18:19:27.516651 13744 ts_tablet_manager.cc:1939] T ad5e4981cb7a47f2b4bf621437f7e96b P bf3cf883769a4bb481dff7f963a2f5f0: Deleting consensus metadata
I20260808 18:19:27.518282 12924 catalog_manager.cc:4841] TS bf3cf883769a4bb481dff7f963a2f5f0 (127.27.18.195:46391): tablet ad5e4981cb7a47f2b4bf621437f7e96b (table test-table [id=86d0684e007742288a2bd55343ae8f67]) successfully deleted
I20260808 18:19:27.529031 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:19:27.500820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33141 (local address 127.0.0.1:40336)
0808 18:19:27.528809 (+ 27989us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.528914 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:33141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27879}
W20260808 18:19:27.529300 12920 catalog_manager.cc:4584] TS 7e2ba73bbdd0403ea363925f8f52d8f7 (127.27.18.193:33141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad5e4981cb7a47f2b4bf621437f7e96b: Network error: Client connection negotiation failed: client connection to 127.27.18.193:33141: connect: Connection refused (error 111)
W20260808 18:19:27.539417 13701 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:19:27.538626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:39147 (local address 127.0.0.1:33294)
0808 18:19:27.538813 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 18:19:27.538829 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:19:27.538928 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:19:27.539284 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.194:39147: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 18:19:27.540171 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 13469
I20260808 18:19:27.553056 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 12894
I20260808 18:19:27.568861 27723 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 18:19:27.913934 27723 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 18:19:27.914054 27723 mini_ranger.cc:66] Stopping Ranger...
I20260808 18:19:28.268281 27723 mini_ranger.cc:68] Stopped Ranger
2026-08-08 18:19:28.268 UTC [12277] LOG: received smart shutdown request
2026-08-08 18:19:28.272 UTC [12277] LOG: background worker "logical replication launcher" (PID 12289) exited with exit code 1
2026-08-08 18:19:28.273 UTC [12281] LOG: shutting down
2026-08-08 18:19:28.420 UTC [12277] LOG: database system is shut down
2026-08-08T18:19:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47507 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:19:28 dist-test-slave-x4qh krb5kdc[13760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:19:28 dist-test-slave-x4qh krb5kdc[13760](info): set up 2 sockets
Aug 08 18:19:28 dist-test-slave-x4qh krb5kdc[13760](info): commencing operation
krb5kdc: starting...
W20260808 18:19:29.767976 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:19:29 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:19:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:19:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/rangeradmin_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/rangeradmin_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/rangerlookup_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/rangerlookup_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
I20260808 18:19:29.925516 27723 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPL43Mv/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/postgres -l logfile start
2026-08-08 18:19:32.118 UTC [13806] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 18:19:32.118 UTC [13806] LOG: listening on IPv4 address "127.27.18.212", port 53687
2026-08-08 18:19:32.121 UTC [13806] LOG: listening on Unix socket "/tmp/.s.PGSQL.53687"
2026-08-08 18:19:32.140 UTC [13809] LOG: database system was shut down at 2026-08-08 18:19:30 UTC
2026-08-08 18:19:32.143 UTC [13806] LOG: database system is ready to accept connections
I20260808 18:19:33.268689 27723 mini_postgres.cc:96] Postgres bound to 53687
2026-08-08 18:19:33.276 UTC [13827] FATAL: database "slave" does not exist
127.27.18.212:53687 - accepting connections
I20260808 18:19:33.277133 27723 mini_ranger.cc:161] Starting Ranger...
I20260808 18:19:33.292639 27723 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 18:19:33.694608 27723 mini_ranger.cc:87] Created ranger Postgres database
I20260808 18:19:33.694711 27723 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-admin
2026-08-08 18:19:33,957 [I] DB FLAVOR :POSTGRES
2026-08-08 18:19:33,957 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:19:33,957 [I] Checking connection..
2026-08-08 18:19:33,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 18:19:34,204 [I] Checking connection passed.
2026-08-08 18:19:34,204 [I] --------- Verifying version history table ---------
2026-08-08 18:19:34,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:19:34,456 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:19:34,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:19:34,695 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 18:19:34,696 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 18:19:34,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 18:19:34.924 UTC [13935] WARNING: there is no transaction in progress
2026-08-08 18:19:34,932 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 18:19:34,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 18:19:35,179 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 18:19:35,179 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:19:35,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 18:19:35,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 18:19:35,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x4qh.c.gcp-upstream.internal','N') ;"
2026-08-08 18:19:35,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:19:36,181 [I] Table x_portal_user does not exist in database ranger
2026-08-08 18:19:36,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:19:36,433 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 18:19:36,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:19:36,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:19:36,911 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 18:19:36,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 18:19:37.803 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.818 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.833 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.847 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.861 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.878 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.936 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.942 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.954 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.963 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.972 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.984 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.992 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:37.998 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:38.004 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:38.010 UTC [14110] WARNING: there is no transaction in progress
2026-08-08 18:19:38,555 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 18:19:38,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 18:19:38,794 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 18:19:38,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 18:19:39,046 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 18:19:39,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 18:19:39,291 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 18:19:39,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 18:19:39,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 18:19:39,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x4qh.c.gcp-upstream.internal';"
2026-08-08 18:19:40,017 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 18:19:40,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 18:19:40,251 [I] DB_PATCHES have already been applied
I20260808 18:19:40.256695 27723 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPL43Mv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 18:19:40.256771 27723 mini_ranger.cc:193] Using host: 127.27.18.212
I20260808 18:19:40.261106 27723 mini_ranger.cc:227] Ranger admin URL: http://127.27.18.212:60961
Aug 08, 2026 6:19:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:19:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:19:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 6:19:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 6:19:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 6:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 6:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 18:19:41.565593 27723 mini_ranger.cc:160] Time spent starting Ranger: real 8.288s user 0.007s sys 0.004s
Aug 08, 2026 6:19:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 6:19:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 6:19:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 6:19:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 6:20:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 18:20:01.241694 27723 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/rangerkms_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/rangerkms_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.18.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.18.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.18.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/HTTP_127.27.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 18:20:01 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213201, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 18:20:01.327262 27723 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 18:20:01.337973 27723 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 18:20:02.090061 27723 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 18:20:02.090155 27723 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms
I20260808 18:20:02.090175 27723 mini_ranger_kms.cc:227] Using postgres at 127.27.18.212:53687
2026-08-08 18:20:02,265 [I] DB FLAVOR :POSTGRES
2026-08-08 18:20:02,265 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 18:20:02,265 [I] Checking connection
2026-08-08 18:20:02,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 18:20:02,517 [I] connection success
2026-08-08 18:20:02,517 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 18:20:02,517 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 18:20:02,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 18:20:02,732 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 18:20:02,732 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 18:20:02,732 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 18:20:02,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.18.212:53687/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 18:20:02,971 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 18:20:03.436453 27723 mini_ranger_kms.cc:310] Created kms service
I20260808 18:20:03.620428 27723 mini_ranger_kms.cc:326] Created kudu user
I20260808 18:20:03.662367 27723 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 18:20:03 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213201, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:04.027176 27723 mini_ranger_kms.cc:384] Added ranger policy
I20260808 18:20:04.027602 27723 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPL43Mv/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPL43Mv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 18:20:04.027650 27723 mini_ranger_kms.cc:240] Using host: 127.27.18.212
I20260808 18:20:04.032829 27723 mini_ranger_kms.cc:276] Ranger KMS PID: 14400
I20260808 18:20:04.032910 27723 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.18.212:57029
Aug 08, 2026 6:20:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 6:20:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 6:20:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 6:20:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 6:20:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 6:20:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 18:20:06 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213206, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.18.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 18:20:06 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213206, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.18.212@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:06.761049 27723 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.434s user 0.008s sys 0.153s
I20260808 18:20:06.761190 27723 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 18:20:06.761260 27723 mini_ranger_kms.cc:401] 127.27.18.212:57029/kms/v1/keys
Aug 08 18:20:06 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213206, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.18.212@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
Aug 08 18:20:09 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:20:09.101940 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44971
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:44971
--ranger_config_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:09.213959 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:09.214252 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:09.214360 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:09.218514 14480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:09.218596 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:09.218626 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:09.218648 14480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:09.218711 14480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:09.224197 14480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.18.212:57029/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:44971
--ranger_config_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:09.225272 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:09.226346 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:09.235417 14490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:09.235348 14492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:09.235337 14489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:09.235934 14480 server_base.cc:1034] running on GCE node
I20260808 18:20:09.236552 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:09.237602 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:09.238798 14480 hybrid_clock.cc:648] HybridClock initialized: now 1786213209238775 us; error 40 us; skew 500 ppm
Aug 08 18:20:09 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213209, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:09.241715 14480 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:09.242622 14480 webserver.cc:460] Webserver started at http://127.27.18.254:35203/ using document root <none> and password file <none>
I20260808 18:20:09.243170 14480 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:09.243255 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:09.243531 14480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:20:09 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213209, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:09.303551 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "4e14c0ee5a5643138d665c3d241ec485"
format_stamp: "Formatted at 2026-08-08 18:20:09 on dist-test-slave-x4qh"
server_key: "4a8895045a85703aa8792147a723c0de"
server_key_iv: "a30c94b21784fc0673edf36e3dbe920f"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:09.304196 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "4e14c0ee5a5643138d665c3d241ec485"
format_stamp: "Formatted at 2026-08-08 18:20:09 on dist-test-slave-x4qh"
server_key: "4a8895045a85703aa8792147a723c0de"
server_key_iv: "a30c94b21784fc0673edf36e3dbe920f"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:09.308243 14480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:20:09.325639 14502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:09.327070 14480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:09.327229 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4e14c0ee5a5643138d665c3d241ec485"
format_stamp: "Formatted at 2026-08-08 18:20:09 on dist-test-slave-x4qh"
server_key: "4a8895045a85703aa8792147a723c0de"
server_key_iv: "a30c94b21784fc0673edf36e3dbe920f"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:09.327363 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:09.339761 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:09.342749 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:09.342970 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:09.351284 14480 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:44971
I20260808 18:20:09.351297 14576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:44971 every 8 connection(s)
I20260808 18:20:09.352376 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:09.355147 14577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:09.360396 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 14480
I20260808 18:20:09.360518 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:20:09.360801 14577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Bootstrap starting.
I20260808 18:20:09.360821 27723 external_mini_cluster.cc:1428] Setting key 60a2bf2e70af5a1082530b6d8d09eaf4
I20260808 18:20:09.363688 14577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:09.364652 14577 log.cc:824] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:09.366946 14577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: No bootstrap required, opened a new log
Aug 08 18:20:09 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:09.371308 14577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:09.371567 14577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:09.371641 14577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e14c0ee5a5643138d665c3d241ec485, State: Initialized, Role: FOLLOWER
I20260808 18:20:09.372104 14577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:09.372252 14577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:09.372326 14577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:09.372413 14577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:09.373394 14577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:09.373726 14577 leader_election.cc:302] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e14c0ee5a5643138d665c3d241ec485; no voters:
I20260808 18:20:09.374006 14577 leader_election.cc:288] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:20:09.374105 14588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:09.374389 14588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 LEADER]: Becoming Leader. State: Replica: 4e14c0ee5a5643138d665c3d241ec485, State: Running, Role: LEADER
I20260808 18:20:09.374748 14588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:09.375267 14577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:09.376134 14588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e14c0ee5a5643138d665c3d241ec485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } } }
I20260808 18:20:09.376307 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:09.362284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.27.18.254:44971)
0808 18:20:09.362786 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:09.362804 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:09.362843 (+ 39us) server_negotiation.cc:407] Connection header received
0808 18:20:09.363691 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:09.363716 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:09.364144 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:09.364563 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:09.365467 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:09.366446 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:09.367512 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:09.367991 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:09.368168 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:09.370833 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:09.370839 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:09.370852 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:09.370877 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:09.372715 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:09.373154 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:09.373161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:09.373171 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:09.373268 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:09.373561 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:09.373564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:09.373566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:09.374046 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:09.374236 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:09.375049 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 18:20:09.375330 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":153,"threads_started":1}
I20260808 18:20:09.376332 14588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:09.376977 14588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e14c0ee5a5643138d665c3d241ec485. Latest consensus state: current_term: 1 leader_uuid: "4e14c0ee5a5643138d665c3d241ec485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } } }
I20260808 18:20:09.377055 14588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:09.377492 14577 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:20:09.381827 14602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:09.386364 14602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:09.398187 14602 catalog_manager.cc:1347] Generated new cluster ID: c4c5bf45b64c496fb0c61d17c00012ae
I20260808 18:20:09.398350 14602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:09.407020 14602 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:20:09.408260 14602 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:09.414235 14602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Generated new TSK 0
I20260808 18:20:09.415072 14602 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:20:10.188009 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44971
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:10.345633 14647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:10.345899 14647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:10.346011 14647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:10.350208 14647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:10.350308 14647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:10.350391 14647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:20:10.356206 14647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.18.212:57029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:10.357177 14647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:10.358130 14647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:10.368135 14657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:10.368160 14654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:10.368391 14653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:10.369336 14647 server_base.cc:1034] running on GCE node
I20260808 18:20:10.369741 14647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:10.370386 14647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:10.371553 14647 hybrid_clock.cc:648] HybridClock initialized: now 1786213210371522 us; error 53 us; skew 500 ppm
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:10.374820 14647 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:20:10.375774 14647 webserver.cc:460] Webserver started at http://127.27.18.193:34297/ using document root <none> and password file <none>
I20260808 18:20:10.376408 14647 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:10.376502 14647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:10.376719 14647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:10.397720 14647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "57fc24c7d47246d381985801da8cf392"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "540da3d8dca2d7a9551b538c26b1be9a"
server_key_iv: "73af558c46c13cbca7caa373249fc96c"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:10.398442 14647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "57fc24c7d47246d381985801da8cf392"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "540da3d8dca2d7a9551b538c26b1be9a"
server_key_iv: "73af558c46c13cbca7caa373249fc96c"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:10.403287 14647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 18:20:10.421028 14667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:10.422782 14647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:10.423017 14647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "57fc24c7d47246d381985801da8cf392"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "540da3d8dca2d7a9551b538c26b1be9a"
server_key_iv: "73af558c46c13cbca7caa373249fc96c"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:10.423272 14647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:10.443518 14647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:10.446511 14647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:10.446702 14647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:10.447474 14647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:10.448483 14647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:10.448550 14647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:10.448639 14647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:10.448689 14647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:10.472651 14647 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:38743
I20260808 18:20:10.474088 14647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:20:10.474553 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 14647
I20260808 18:20:10.474695 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:20:10.475055 27723 external_mini_cluster.cc:1428] Setting key 7e2789f2f688fd837f3179a60c9b94b0
I20260808 18:20:10.479753 14855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:38743 every 8 connection(s)
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:10.497390 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:10.482681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44971 (local address 127.27.18.193:48547)
0808 18:20:10.484399 (+ 1718us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:10.484440 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:10.485487 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:10.486173 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:10.486182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:10.486551 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:10.487104 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:10.487118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.487974 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.487978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:10.488881 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:10.488890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.489419 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.489423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:10.489636 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:10.490309 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:10.490325 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:10.491916 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:10.493817 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:10.493822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:10.493835 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:10.494196 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:10.494487 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:10.494491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:10.494493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:10.494642 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:10.495040 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:10.495046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:10.495423 (+ 377us) client_negotiation.cc:769] Sending connection context
0808 18:20:10.495673 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 18:20:10.496047 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"thread_start_us":263,"threads_started":1}
I20260808 18:20:10.498312 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:10.483861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:48547 (local address 127.27.18.254:44971)
0808 18:20:10.484144 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:10.484148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:10.484539 (+ 391us) server_negotiation.cc:407] Connection header received
0808 18:20:10.485690 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:10.485694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:10.485746 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:10.485855 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:10.487249 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.487853 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.489004 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.489270 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.489383 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:10.492076 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:10.492080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:10.492082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:10.492103 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:10.493687 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:10.494302 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:10.494306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:10.494307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:10.494391 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:10.494742 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:10.494747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:10.494748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:10.494931 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:10.495040 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:10.497941 (+ 2901us) server_negotiation.cc:299] Negotiation successful
0808 18:20:10.498110 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":87,"threads_started":1}
I20260808 18:20:10.500385 14856 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44971
I20260808 18:20:10.500728 14856 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:20:10.501379 14856 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:10.502969 14527 ts_manager.cc:194] Registered new tserver with Master: 57fc24c7d47246d381985801da8cf392 (127.27.18.193:38743)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:20:10.504417 14527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:48547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:20:10.532830 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44971
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 18:20:10.694468 14866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:10.694716 14866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:10.694841 14866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:10.700404 14866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:10.700515 14866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:10.700645 14866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:20:10.707942 14866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.18.212:57029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:10.709851 14866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:10.711151 14866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:10.722476 14877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:10.723441 14866 server_base.cc:1034] running on GCE node
W20260808 18:20:10.723971 14880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:10.723985 14878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:10.724527 14866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:10.725098 14866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:10.728376 14866 hybrid_clock.cc:648] HybridClock initialized: now 1786213210728343 us; error 51 us; skew 500 ppm
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:10.731529 14866 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:20:10.732417 14866 webserver.cc:460] Webserver started at http://127.27.18.194:36457/ using document root <none> and password file <none>
I20260808 18:20:10.733042 14866 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:10.733124 14866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:10.733419 14866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:10.752393 14866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "ae4c87ea29e14914a2fa6f64aeaa9646"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "0da1d3cfb821d30368ef6fbd8678cfac"
server_key_iv: "2e56a816a041d42f4895c72637327ca5"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:10.753211 14866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "ae4c87ea29e14914a2fa6f64aeaa9646"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "0da1d3cfb821d30368ef6fbd8678cfac"
server_key_iv: "2e56a816a041d42f4895c72637327ca5"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:10.758507 14866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 18:20:10.776502 14895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:10.778019 14866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:20:10.778141 14866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "ae4c87ea29e14914a2fa6f64aeaa9646"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "0da1d3cfb821d30368ef6fbd8678cfac"
server_key_iv: "2e56a816a041d42f4895c72637327ca5"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:10.778278 14866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:10.794646 14866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:10.798477 14866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:10.798708 14866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:10.799436 14866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:10.800570 14866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:10.800684 14866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:10.800792 14866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:10.800871 14866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:10.813489 14866 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:40925
I20260808 18:20:10.814323 15076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:40925 every 8 connection(s)
I20260808 18:20:10.814455 14866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:20:10.824420 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 14866
I20260808 18:20:10.824585 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:20:10.824910 27723 external_mini_cluster.cc:1428] Setting key 278bf9e5920bf92942c54597ac52e586
Aug 08 18:20:10 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:10.830444 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:10.816499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:52037 (local address 127.27.18.254:44971)
0808 18:20:10.816688 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:10.816693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:10.817594 (+ 901us) server_negotiation.cc:407] Connection header received
0808 18:20:10.818776 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:10.818779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:10.818832 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:10.818906 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:10.820385 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.820900 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.822656 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.823018 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.823129 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:10.826163 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:10.826170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:10.826182 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:10.826213 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:10.828147 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:10.828800 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:10.828806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:10.828811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:10.828912 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:10.829290 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:10.829294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:10.829296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:10.829479 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:10.829589 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:10.830080 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 18:20:10.830209 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:20:10.831357 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:10.816904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44971 (local address 127.27.18.194:52037)
0808 18:20:10.817408 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:10.817469 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:10.818527 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:10.819130 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:10.819140 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:10.819563 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:10.820229 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:10.820251 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.821060 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.821063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:10.822518 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:10.822530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:10.823212 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:10.823217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:10.823451 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:10.824149 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:10.824172 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:10.825981 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:10.828275 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:10.828282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:10.828298 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:10.828673 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:10.829036 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:10.829040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:10.829042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:10.829183 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:10.829589 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:10.829596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:10.829826 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 18:20:10.830039 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 18:20:10.830344 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":106,"threads_started":1}
I20260808 18:20:10.832669 15077 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44971
I20260808 18:20:10.832923 15077 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:10.833580 15077 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:10.835189 14527 ts_manager.cc:194] Registered new tserver with Master: ae4c87ea29e14914a2fa6f64aeaa9646 (127.27.18.194:40925)
I20260808 18:20:10.836035 14527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:52037
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:20:10.856297 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44971
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:11.072577 15087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:11.072962 15087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:11.073079 15087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:11.078431 15087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:11.078500 15087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:11.078574 15087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:20:11.085419 15087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.18.212:57029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:11.086764 15087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:11.088095 15087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:11.100495 15093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:11.100544 15095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:11.100919 15087 server_base.cc:1034] running on GCE node
W20260808 18:20:11.101068 15098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:11.101482 15087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:11.102241 15087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:11.103461 15087 hybrid_clock.cc:648] HybridClock initialized: now 1786213211103436 us; error 38 us; skew 500 ppm
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:11.107347 15087 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:20:11.108372 15087 webserver.cc:460] Webserver started at http://127.27.18.195:37195/ using document root <none> and password file <none>
I20260808 18:20:11.109150 15087 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:11.109244 15087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:11.109495 15087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:11.129956 15087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "0b0af7c14bd34312beac2f75633d8a0e"
format_stamp: "Formatted at 2026-08-08 18:20:11 on dist-test-slave-x4qh"
server_key: "13630a02ac883e750c286e53c9c6e80f"
server_key_iv: "0f219e20b33c07b6f7985adece7f52b2"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:11.130769 15087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "0b0af7c14bd34312beac2f75633d8a0e"
format_stamp: "Formatted at 2026-08-08 18:20:11 on dist-test-slave-x4qh"
server_key: "13630a02ac883e750c286e53c9c6e80f"
server_key_iv: "0f219e20b33c07b6f7985adece7f52b2"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:11.138669 15087 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.001s
I20260808 18:20:11.163717 15108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.165482 15087 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.001s
I20260808 18:20:11.165628 15087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "0b0af7c14bd34312beac2f75633d8a0e"
format_stamp: "Formatted at 2026-08-08 18:20:11 on dist-test-slave-x4qh"
server_key: "13630a02ac883e750c286e53c9c6e80f"
server_key_iv: "0f219e20b33c07b6f7985adece7f52b2"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:11.165750 15087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:11.179203 15087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:11.182288 15087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:11.182478 15087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:11.183073 15087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:11.184088 15087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:11.184196 15087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.184437 15087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:11.184528 15087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.195313 15087 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46203
I20260808 18:20:11.196409 15087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:20:11.197362 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 15087
I20260808 18:20:11.197481 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:20:11.197767 27723 external_mini_cluster.cc:1428] Setting key 3949202886a2145f26024479e3ecc225
I20260808 18:20:11.200142 15300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46203 every 8 connection(s)
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:11.217722 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:11.203756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:55619 (local address 127.27.18.254:44971)
0808 18:20:11.203864 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:11.203869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:11.204276 (+ 407us) server_negotiation.cc:407] Connection header received
0808 18:20:11.205694 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:11.205697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:11.205748 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:11.205796 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:11.207271 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.208049 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.209622 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.209954 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.210022 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:11.213399 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:11.213403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:11.213406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:11.213433 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:11.215187 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:11.215898 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:11.215901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:11.215903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:11.215998 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:11.216489 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:11.216493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:11.216495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:11.216717 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:11.216803 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:11.217445 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 18:20:11.217571 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:20:11.219460 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:11.202747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44971 (local address 127.27.18.195:55619)
0808 18:20:11.204131 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:11.204190 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:11.205445 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:11.206168 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:11.206176 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:11.206513 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:11.207108 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:11.207124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.208202 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.208204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:11.209485 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:11.209495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.210414 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.210418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:11.210606 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:11.211403 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:11.211424 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:11.213211 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:11.215328 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:11.215334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:11.215364 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:11.215770 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:11.216134 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:11.216137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:11.216141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:11.216322 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:11.216872 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:11.216878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:11.217172 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 18:20:11.218137 (+ 965us) client_negotiation.cc:240] Negotiation successful
0808 18:20:11.218444 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":143,"threads_started":1}
I20260808 18:20:11.220887 15303 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44971
I20260808 18:20:11.221143 15303 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:11.221624 15303 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:11.223757 14527 ts_manager.cc:194] Registered new tserver with Master: 0b0af7c14bd34312beac2f75633d8a0e (127.27.18.195:46203)
I20260808 18:20:11.224495 14527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:55619
I20260808 18:20:11.234982 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:20:11.235958 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 14647
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213210, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:11.270406 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 14866
I20260808 18:20:11.280277 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 15087
I20260808 18:20:11.373724 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 14480
I20260808 18:20:11.385653 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44971
--webserver_interface=127.27.18.254
--webserver_port=35203
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:44971
--ranger_config_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:11.491616 15310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:11.491834 15310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:11.491894 15310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:11.495745 15310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:11.495792 15310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:11.495810 15310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:11.495826 15310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:11.495873 15310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:11.500860 15310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:44971
--ranger_config_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=35203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:11.501708 15310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:11.502579 15310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:11.510401 15321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:11.510401 15317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:11.510769 15318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:11.511211 15310 server_base.cc:1034] running on GCE node
I20260808 18:20:11.511860 15310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:11.513087 15310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:11.514247 15310 hybrid_clock.cc:648] HybridClock initialized: now 1786213211514231 us; error 28 us; skew 500 ppm
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:11.516907 15310 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:11.517729 15310 webserver.cc:460] Webserver started at http://127.27.18.254:35203/ using document root <none> and password file <none>
I20260808 18:20:11.518203 15310 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:11.518308 15310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:11.595922 15310 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:20:11.598037 15340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.599218 15310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:11.599345 15310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4e14c0ee5a5643138d665c3d241ec485"
format_stamp: "Formatted at 2026-08-08 18:20:09 on dist-test-slave-x4qh"
server_key: "4a8895045a85703aa8792147a723c0de"
server_key_iv: "a30c94b21784fc0673edf36e3dbe920f"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:11.599890 15310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:11.635182 15310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:11.637665 15310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:11.637845 15310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:11.644878 15310 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:44971
I20260808 18:20:11.644888 15424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:44971 every 8 connection(s)
I20260808 18:20:11.645910 15310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:11.649942 15428 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:20:11.651474 15428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Bootstrap starting.
I20260808 18:20:11.653514 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 15310
I20260808 18:20:11.653973 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:38743
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:44971
--webserver_port=34297
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:20:11.667459 15428 log.cc:824] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:11.673563 15428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:20:11.673913 15428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Bootstrap complete.
I20260808 18:20:11.677768 15428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:11.678153 15428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e14c0ee5a5643138d665c3d241ec485, State: Initialized, Role: FOLLOWER
I20260808 18:20:11.678649 15428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:11.678790 15428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:11.678886 15428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:11.679001 15428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:20:11.680756 15428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:11.681077 15428 leader_election.cc:302] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e14c0ee5a5643138d665c3d241ec485; no voters:
I20260808 18:20:11.681360 15428 leader_election.cc:288] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:20:11.681479 15432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:20:11.681685 15432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [term 2 LEADER]: Becoming Leader. State: Replica: 4e14c0ee5a5643138d665c3d241ec485, State: Running, Role: LEADER
I20260808 18:20:11.682034 15432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } }
I20260808 18:20:11.682533 15428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:11.682932 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e14c0ee5a5643138d665c3d241ec485. Latest consensus state: current_term: 2 leader_uuid: "4e14c0ee5a5643138d665c3d241ec485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } } }
I20260808 18:20:11.683113 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:11.683313 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4e14c0ee5a5643138d665c3d241ec485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e14c0ee5a5643138d665c3d241ec485" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44971 } } }
I20260808 18:20:11.683449 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:11.683621 15436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:11.686724 15436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:11.687196 15428 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 18:20:11.687412 15436 catalog_manager.cc:1259] Loaded cluster ID: c4c5bf45b64c496fb0c61d17c00012ae
I20260808 18:20:11.687455 15436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:11.689764 15436 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:11.690440 15436 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Loaded TSK: 0
I20260808 18:20:11.691618 15436 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:20:11.787220 15430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:11.787497 15430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:11.787580 15430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:11.791452 15430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:11.791518 15430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:11.791606 15430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:20:11.797338 15430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:38743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=34297
--tserver_master_addrs=127.27.18.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:11.798592 15430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:11.799877 15430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:11.810262 15484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:11.810213 15487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:11.810222 15481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:11.810735 15430 server_base.cc:1034] running on GCE node
I20260808 18:20:11.811115 15430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:11.811851 15430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:11.813076 15430 hybrid_clock.cc:648] HybridClock initialized: now 1786213211813046 us; error 40 us; skew 500 ppm
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:11.815831 15430 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:20:11.816777 15430 webserver.cc:460] Webserver started at http://127.27.18.193:34297/ using document root <none> and password file <none>
I20260808 18:20:11.817370 15430 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:11.817529 15430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:11.841660 15430 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:11.845112 15496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.848456 15430 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260808 18:20:11.848631 15430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "57fc24c7d47246d381985801da8cf392"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "540da3d8dca2d7a9551b538c26b1be9a"
server_key_iv: "73af558c46c13cbca7caa373249fc96c"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:11.849471 15430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:11.877262 15430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:11.880782 15430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:11.881006 15430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:11.881728 15430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:11.882895 15430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:11.883031 15430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.883124 15430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:11.883183 15430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:11.895422 15430 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:38743
I20260808 18:20:11.895452 15674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:38743 every 8 connection(s)
I20260808 18:20:11.896553 15430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:20:11.901836 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 15430
I20260808 18:20:11.902390 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:40925
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:44971
--webserver_port=36457
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:20:11 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213211, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:11.918067 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:11.898804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:46885 (local address 127.27.18.254:44971)
0808 18:20:11.900818 (+ 2014us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:11.900839 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:11.900873 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:20:11.901087 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:11.901109 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:11.901378 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:11.901697 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:11.903525 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.905204 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.906961 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.907815 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.907982 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:11.911333 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:11.911339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:11.911354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:11.911406 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:11.913336 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:11.914057 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:11.914065 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:11.914072 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:11.914230 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:11.914806 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:11.914812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:11.914816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:11.915368 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:11.915545 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:11.916435 (+ 890us) server_negotiation.cc:299] Negotiation successful
0808 18:20:11.916858 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":145,"threads_started":1}
I20260808 18:20:11.919906 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:11.898946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44971 (local address 127.27.18.193:46885)
0808 18:20:11.899638 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:11.899687 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:11.900506 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:11.902232 (+ 1726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:11.902240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:11.902584 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:11.903230 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:11.903254 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.905349 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.905354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:11.906788 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:11.906813 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:11.908273 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:11.908280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:11.908533 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:11.909195 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:11.909218 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:11.911101 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:11.913474 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:11.913483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:11.913497 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:11.913889 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:11.914400 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:11.914406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:11.914411 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:11.914657 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:11.915681 (+ 1024us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:11.915690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:11.916017 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 18:20:11.918724 (+ 2707us) client_negotiation.cc:240] Negotiation successful
0808 18:20:11.919020 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":149,"threads_started":1}
I20260808 18:20:11.922610 15675 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44971
I20260808 18:20:11.922852 15675 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:11.923455 15675 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
W20260808 18:20:11.925593 15675 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:44971 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:20:11.925726 15675 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:11.925911 15675 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:11.926807 15675 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:11.926997 15675 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
W20260808 18:20:11.927845 15675 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:20:12.060504 15680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:12.060882 15680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:12.060956 15680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:12.066920 15680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:12.066993 15680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:12.067088 15680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:20:12.074648 15680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:40925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=36457
--tserver_master_addrs=127.27.18.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:12.076105 15680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:12.077469 15680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:12.088269 15688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:12.088399 15692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:12.088269 15690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:12.088896 15680 server_base.cc:1034] running on GCE node
I20260808 18:20:12.089239 15680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:12.089802 15680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:12.090975 15680 hybrid_clock.cc:648] HybridClock initialized: now 1786213212090951 us; error 40 us; skew 500 ppm
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:12.093552 15680 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:20:12.094435 15680 webserver.cc:460] Webserver started at http://127.27.18.194:36457/ using document root <none> and password file <none>
I20260808 18:20:12.094982 15680 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:12.095083 15680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:12.119572 15680 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 18:20:12.126544 15703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:12.128227 15680 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260808 18:20:12.128353 15680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "ae4c87ea29e14914a2fa6f64aeaa9646"
format_stamp: "Formatted at 2026-08-08 18:20:10 on dist-test-slave-x4qh"
server_key: "0da1d3cfb821d30368ef6fbd8678cfac"
server_key_iv: "2e56a816a041d42f4895c72637327ca5"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:12.129021 15680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:12.155151 15680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:12.158799 15680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:12.159066 15680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:12.159837 15680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:12.160926 15680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:12.161042 15680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:12.161213 15680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:12.161295 15680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:12.174870 15680 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:40925
I20260808 18:20:12.176219 15680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:20:12.177155 15900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:40925 every 8 connection(s)
I20260808 18:20:12.183238 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 15680
I20260808 18:20:12.183743 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.18.212:57029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:46203
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:44971
--webserver_port=37195
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:12.209157 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:12.181137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:54377 (local address 127.27.18.254:44971)
0808 18:20:12.181289 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:12.181295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:12.186212 (+ 4917us) server_negotiation.cc:407] Connection header received
0808 18:20:12.189976 (+ 3764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:12.189981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:12.190053 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:12.190113 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:12.198036 (+ 7923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.198815 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.200409 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.200751 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.200822 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:12.205103 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:12.205108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:12.205110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:12.205138 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:12.206843 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:12.207440 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:12.207443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:12.207445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:12.207531 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:12.207913 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:12.207916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:12.207918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:12.208134 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:12.208216 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:12.208843 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 18:20:12.208984 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 18:20:12.210608 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:12.180500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44971 (local address 127.27.18.194:54377)
0808 18:20:12.186030 (+ 5530us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:12.186093 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:12.189699 (+ 3606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:12.190811 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:12.190822 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:12.191203 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:12.191995 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:12.192018 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.198967 (+ 6949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.198970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:12.200270 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:12.200282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.201020 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.201024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:12.201204 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:12.203183 (+ 1979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:12.203203 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:12.204894 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:12.206951 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:12.206957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:12.206971 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:12.207292 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:12.207639 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:12.207641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:12.207643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:12.207810 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:12.208283 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:12.208289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:12.208566 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 18:20:12.209328 (+ 762us) client_negotiation.cc:240] Negotiation successful
0808 18:20:12.209635 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5278,"thread_start_us":146,"threads_started":1}
I20260808 18:20:12.212031 15902 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44971
I20260808 18:20:12.212342 15902 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.212903 15902 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
W20260808 18:20:12.214558 15902 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:44971 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:20:12.214679 15902 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.214831 15902 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:12.215667 15902 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.215847 15902 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
W20260808 18:20:12.216645 15902 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:20:12.345338 15906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:12.345662 15906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:12.345729 15906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:12.351622 15906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:12.351713 15906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:12.351805 15906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:20:12.359002 15906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.18.212:57029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:46203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=37195
--tserver_master_addrs=127.27.18.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:12.360201 15906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:12.361533 15906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:12.372668 15917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:12.372924 15906 server_base.cc:1034] running on GCE node
W20260808 18:20:12.372973 15922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:12.373185 15915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:12.373611 15906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:12.374259 15906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:12.375438 15906 hybrid_clock.cc:648] HybridClock initialized: now 1786213212375415 us; error 31 us; skew 500 ppm
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:12.378506 15906 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:20:12.379554 15906 webserver.cc:460] Webserver started at http://127.27.18.195:37195/ using document root <none> and password file <none>
I20260808 18:20:12.380250 15906 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:12.380375 15906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:12.401290 15906 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 18:20:12.404022 15934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:12.405393 15906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:12.405562 15906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "0b0af7c14bd34312beac2f75633d8a0e"
format_stamp: "Formatted at 2026-08-08 18:20:11 on dist-test-slave-x4qh"
server_key: "13630a02ac883e750c286e53c9c6e80f"
server_key_iv: "0f219e20b33c07b6f7985adece7f52b2"
server_key_version: "kuduclusterkey@0"
I20260808 18:20:12.406126 15906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:12.421830 15906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:12.425028 15906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:12.425262 15906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:12.425956 15906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:12.426885 15906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:12.426935 15906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:12.426999 15906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:12.427029 15906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:12.438088 15906 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46203
I20260808 18:20:12.439199 15906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:20:12.443706 16134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46203 every 8 connection(s)
I20260808 18:20:12.444849 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 15906
I20260808 18:20:12.462260 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:12.445638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.27.18.254:44971)
0808 18:20:12.445774 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:12.445778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:12.446095 (+ 317us) server_negotiation.cc:407] Connection header received
0808 18:20:12.446203 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:12.446207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:12.446269 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:12.446370 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:12.447333 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.448173 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.449334 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.449688 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.449796 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:12.451401 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:12.451405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:12.451408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:12.451435 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:12.459939 (+ 8504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:12.460437 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:12.460443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:12.460445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:12.460537 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:12.461293 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:12.461297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:12.461300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:12.461532 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:12.461768 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:12.461953 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 18:20:12.462068 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:12.468082 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:12.447418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:37779 (local address 127.27.18.254:44971)
0808 18:20:12.447648 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:12.447656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:12.447769 (+ 113us) server_negotiation.cc:407] Connection header received
0808 18:20:12.448796 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:12.448800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:12.448850 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:12.448947 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:12.450193 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.450976 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.460341 (+ 9365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.460585 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.460706 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:12.464198 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:12.464203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:12.464206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:12.464231 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:12.465788 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:12.466312 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:12.466316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:12.466317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:12.466427 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:12.466813 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:12.466816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:12.466818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:12.467028 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:12.467136 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:12.467802 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 18:20:12.467913 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
I20260808 18:20:12.469007 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:12.446297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:44971 (local address 127.27.18.195:37779)
0808 18:20:12.447615 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:12.447680 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:12.448557 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:12.449131 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:12.449140 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:12.449459 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:12.450019 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:12.450043 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.451155 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.451158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:12.460195 (+ 9037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:12.460208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:12.460882 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:12.460885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:12.461021 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:12.462424 (+ 1403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:12.462445 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:12.464042 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:12.465920 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:12.465926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:12.465942 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:12.466198 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:12.466525 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:12.466528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:12.466530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:12.466707 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:12.467136 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:12.467144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:12.467527 (+ 383us) client_negotiation.cc:769] Sending connection context
0808 18:20:12.467749 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 18:20:12.468041 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"thread_start_us":144,"threads_started":1}
I20260808 18:20:12.470366 16136 heartbeater.cc:347] Connected to a master server at 127.27.18.254:44971
I20260808 18:20:12.470655 16136 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.471293 16136 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
W20260808 18:20:12.473060 16136 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:44971 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 18:20:12.473201 16136 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.473382 16136 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:12.474288 16136 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.474454 16136 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
W20260808 18:20:12.475181 16136 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 18:20:12 dist-test-slave-x4qh krb5kdc[13760](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:12.928092 15675 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:12.928395 15675 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:12.929579 15366 ts_manager.cc:194] Registered new tserver with Master: 57fc24c7d47246d381985801da8cf392 (127.27.18.193:38743)
I20260808 18:20:12.931259 15366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:46885
I20260808 18:20:13.216883 15902 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:13.217173 15902 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:13.217952 15366 ts_manager.cc:194] Registered new tserver with Master: ae4c87ea29e14914a2fa6f64aeaa9646 (127.27.18.194:40925)
I20260808 18:20:13.218679 15366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:54377
Aug 08 18:20:13 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
Aug 08 18:20:13 dist-test-slave-x4qh krb5kdc[13760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213212, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for HTTP/127.27.18.212@KRBTEST.COM
I20260808 18:20:13.475448 16136 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:13.475857 16136 heartbeater.cc:511] Master 127.27.18.254:44971 requested a full tablet report, sending...
I20260808 18:20:13.476785 15366 ts_manager.cc:194] Registered new tserver with Master: 0b0af7c14bd34312beac2f75633d8a0e (127.27.18.195:46203)
I20260808 18:20:13.477672 15366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:37779
I20260808 18:20:13.481678 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:20:13.501329 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.485453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.27.18.254:44971)
0808 18:20:13.485975 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.485980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.486303 (+ 323us) server_negotiation.cc:407] Connection header received
0808 18:20:13.486346 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.486349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.486392 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.486464 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:13.489502 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.490145 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.491562 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.491947 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.492065 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.495617 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:13.495621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:13.495624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:13.495654 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:13.497751 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:13.498245 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:13.498250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:13.498252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:13.498359 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:13.498718 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:13.498722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:13.498723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:13.498957 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:13.499108 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.499842 (+ 734us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.501132 (+ 1290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":64,"threads_started":1}
I20260808 18:20:13.510622 15366 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:20:13.519304 15366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:20:13.561702 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.541014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:40925 (local address 127.0.0.1:52104)
0808 18:20:13.544142 (+ 3128us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:13.544152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:13.544212 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:13.547255 (+ 3043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:13.547258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:13.547274 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:13.547351 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.547356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.552514 (+ 5158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.552517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.553643 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.553654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.556698 (+ 3044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.556702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.556814 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.556854 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 18:20:13.556961 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 18:20:13.557102 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3079,"thread_start_us":91,"threads_started":1}
I20260808 18:20:13.563478 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.540273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46203 (local address 127.0.0.1:33920)
0808 18:20:13.543804 (+ 3531us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:13.543849 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:13.543960 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:13.545059 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:13.545065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:13.545087 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:13.545208 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.545214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.548292 (+ 3078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.548295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.550242 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.550253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.563013 (+ 12760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.563016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.563137 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.563161 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:20:13.563240 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:20:13.563321 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3367,"thread_start_us":94,"threads_started":1}
I20260808 18:20:13.564289 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.541555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.27.18.195:46203)
0808 18:20:13.544454 (+ 2899us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.544461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.544483 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:20:13.544553 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.544558 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.544759 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.544892 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:13.545468 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.548167 (+ 2699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.554518 (+ 6351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.562719 (+ 8201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.562819 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.562952 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.564040 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.564148 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2763,"thread_start_us":126,"threads_started":1}
I20260808 18:20:13.565618 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.541913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.27.18.194:40925)
0808 18:20:13.545988 (+ 4075us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.545994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.546015 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:20:13.546801 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.546813 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.547008 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.547132 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:13.550382 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.552377 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.555603 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.556375 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.556478 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.556623 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.565378 (+ 8755us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.565482 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3980,"thread_start_us":96,"threads_started":1}
I20260808 18:20:13.567425 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.543241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38743 (local address 127.0.0.1:55682)
0808 18:20:13.562346 (+ 19105us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:13.562360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:13.562447 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:13.563158 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:13.563161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:13.563180 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:13.563281 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.563287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.564790 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.564793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.565818 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.565828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.567039 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.567043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.567127 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.567149 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:20:13.567218 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 18:20:13.567281 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19048,"thread_start_us":88,"threads_started":1}
I20260808 18:20:13.568075 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.543417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.27.18.193:38743)
0808 18:20:13.546772 (+ 3355us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.546778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.562469 (+ 15691us) server_negotiation.cc:407] Connection header received
0808 18:20:13.562578 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.562584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.562794 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.562950 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:13.563464 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.564654 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.565960 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.566728 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.566832 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.566978 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.567842 (+ 864us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.567935 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3252,"thread_start_us":103,"threads_started":1}
I20260808 18:20:13.567687 15988 tablet_service.cc:1467] Processing CreateTablet for tablet ebf75243462f4012a10c2d478cefdb86 (DEFAULT_TABLE table=test-table [id=5d61d92b204448ae8e776db395454d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:13.569044 15988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf75243462f4012a10c2d478cefdb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:13.569526 15797 tablet_service.cc:1467] Processing CreateTablet for tablet ebf75243462f4012a10c2d478cefdb86 (DEFAULT_TABLE table=test-table [id=5d61d92b204448ae8e776db395454d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:13.570637 15797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf75243462f4012a10c2d478cefdb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:13.570900 15558 tablet_service.cc:1467] Processing CreateTablet for tablet ebf75243462f4012a10c2d478cefdb86 (DEFAULT_TABLE table=test-table [id=5d61d92b204448ae8e776db395454d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:13.571879 15558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebf75243462f4012a10c2d478cefdb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:13.577272 16192 tablet_bootstrap.cc:492] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Bootstrap starting.
I20260808 18:20:13.579615 16192 tablet_bootstrap.cc:654] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:13.580514 16192 log.cc:824] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:13.583719 16194 tablet_bootstrap.cc:492] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Bootstrap starting.
I20260808 18:20:13.586880 16193 tablet_bootstrap.cc:492] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: Bootstrap starting.
I20260808 18:20:13.593652 16193 tablet_bootstrap.cc:654] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:13.597822 16192 tablet_bootstrap.cc:492] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: No bootstrap required, opened a new log
I20260808 18:20:13.598100 16192 ts_tablet_manager.cc:1397] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.005s
I20260808 18:20:13.602156 16192 raft_consensus.cc:348] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.604485 16194 tablet_bootstrap.cc:654] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:13.610467 16194 log.cc:824] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:13.611938 16193 log.cc:824] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:13.612439 16192 raft_consensus.cc:374] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:13.612569 16192 raft_consensus.cc:729] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0af7c14bd34312beac2f75633d8a0e, State: Initialized, Role: FOLLOWER
I20260808 18:20:13.613186 16192 consensus_queue.cc:260] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.614464 16192 ts_tablet_manager.cc:1428] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Time spent starting tablet: real 0.016s user 0.000s sys 0.005s
I20260808 18:20:13.614903 16136 heartbeater.cc:503] Master 127.27.18.254:44971 was elected leader, sending a full tablet report...
I20260808 18:20:13.629901 16194 tablet_bootstrap.cc:492] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: No bootstrap required, opened a new log
I20260808 18:20:13.630223 16194 ts_tablet_manager.cc:1397] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.000s
I20260808 18:20:13.636624 16193 tablet_bootstrap.cc:492] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: No bootstrap required, opened a new log
I20260808 18:20:13.636924 16193 ts_tablet_manager.cc:1397] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.003s
I20260808 18:20:13.640987 16193 raft_consensus.cc:348] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.642351 16194 raft_consensus.cc:348] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.644578 16194 raft_consensus.cc:374] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:13.644692 16194 raft_consensus.cc:729] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c87ea29e14914a2fa6f64aeaa9646, State: Initialized, Role: FOLLOWER
I20260808 18:20:13.645249 16194 consensus_queue.cc:260] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.646421 16194 ts_tablet_manager.cc:1428] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260808 18:20:13.646545 16193 raft_consensus.cc:374] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:13.646641 16193 raft_consensus.cc:729] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57fc24c7d47246d381985801da8cf392, State: Initialized, Role: FOLLOWER
I20260808 18:20:13.647187 16193 consensus_queue.cc:260] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.648264 16193 ts_tablet_manager.cc:1428] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: Time spent starting tablet: real 0.011s user 0.003s sys 0.002s
I20260808 18:20:13.648665 15902 heartbeater.cc:503] Master 127.27.18.254:44971 was elected leader, sending a full tablet report...
W20260808 18:20:13.651221 15676 tablet.cc:2369] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:20:13.651489 15675 heartbeater.cc:503] Master 127.27.18.254:44971 was elected leader, sending a full tablet report...
W20260808 18:20:13.680446 15904 tablet.cc:2369] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:20:13.695147 16137 tablet.cc:2369] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:20:13.713918 16201 raft_consensus.cc:482] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:13.714148 16201 raft_consensus.cc:504] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.715444 16201 leader_election.cc:288] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae4c87ea29e14914a2fa6f64aeaa9646 (127.27.18.194:40925), 0b0af7c14bd34312beac2f75633d8a0e (127.27.18.195:46203)
I20260808 18:20:13.721161 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.716032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:47115 (local address 127.27.18.194:40925)
0808 18:20:13.716182 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.716188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.716333 (+ 145us) server_negotiation.cc:407] Connection header received
0808 18:20:13.716511 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.716514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.716561 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.716609 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:13.717091 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.717862 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.719742 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.720321 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.720413 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.720464 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.720961 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.721036 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:20:13.721161 16186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.716047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:49987 (local address 127.27.18.195:46203)
0808 18:20:13.716293 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.716298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.716312 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:20:13.716512 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.716515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.716561 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.716609 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:13.717091 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.717857 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.719742 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.720329 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.720431 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.720471 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.720961 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.721036 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260808 18:20:13.721693 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.715615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:40925 (local address 127.27.18.193:47115)
0808 18:20:13.716239 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:13.716255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:13.716383 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:13.716753 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:13.716756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:13.716843 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:13.716949 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.716956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.718020 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.718023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.719600 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.719608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.720565 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.720569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.720788 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.720842 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 18:20:13.721450 (+ 608us) client_negotiation.cc:240] Negotiation successful
0808 18:20:13.721530 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":260,"spinlock_wait_cycles":19328,"thread_start_us":126,"threads_started":1}
I20260808 18:20:13.721702 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.715747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46203 (local address 127.27.18.193:49987)
0808 18:20:13.716239 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:13.716255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:13.716382 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:13.716753 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:13.716756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:13.716774 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:13.716949 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.716956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.718020 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.718023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.719597 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:13.719608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.720565 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.720569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:13.720775 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.720836 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 18:20:13.721497 (+ 661us) client_negotiation.cc:240] Negotiation successful
0808 18:20:13.721543 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":259,"spinlock_wait_cycles":24320,"thread_start_us":69,"threads_started":1}
I20260808 18:20:13.722414 15804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebf75243462f4012a10c2d478cefdb86" candidate_uuid: "57fc24c7d47246d381985801da8cf392" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" is_pre_election: true
I20260808 18:20:13.722442 16029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebf75243462f4012a10c2d478cefdb86" candidate_uuid: "57fc24c7d47246d381985801da8cf392" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0af7c14bd34312beac2f75633d8a0e" is_pre_election: true
I20260808 18:20:13.722712 16029 raft_consensus.cc:2454] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57fc24c7d47246d381985801da8cf392 in term 0.
I20260808 18:20:13.722718 15804 raft_consensus.cc:2454] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57fc24c7d47246d381985801da8cf392 in term 0.
I20260808 18:20:13.723186 15498 leader_election.cc:302] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57fc24c7d47246d381985801da8cf392, ae4c87ea29e14914a2fa6f64aeaa9646; no voters:
I20260808 18:20:13.723697 16201 raft_consensus.cc:2781] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:20:13.723788 16201 raft_consensus.cc:482] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:20:13.723841 16201 raft_consensus.cc:3037] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:13.724864 16201 raft_consensus.cc:504] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.725190 16201 leader_election.cc:288] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [CANDIDATE]: Term 1 election: Requested vote from peers ae4c87ea29e14914a2fa6f64aeaa9646 (127.27.18.194:40925), 0b0af7c14bd34312beac2f75633d8a0e (127.27.18.195:46203)
I20260808 18:20:13.725500 15804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebf75243462f4012a10c2d478cefdb86" candidate_uuid: "57fc24c7d47246d381985801da8cf392" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646"
I20260808 18:20:13.725630 15804 raft_consensus.cc:3037] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:13.726379 16029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebf75243462f4012a10c2d478cefdb86" candidate_uuid: "57fc24c7d47246d381985801da8cf392" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0af7c14bd34312beac2f75633d8a0e"
I20260808 18:20:13.726492 16029 raft_consensus.cc:3037] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:13.727174 15804 raft_consensus.cc:2454] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57fc24c7d47246d381985801da8cf392 in term 1.
I20260808 18:20:13.727512 15498 leader_election.cc:302] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57fc24c7d47246d381985801da8cf392, ae4c87ea29e14914a2fa6f64aeaa9646; no voters:
I20260808 18:20:13.727991 16029 raft_consensus.cc:2454] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57fc24c7d47246d381985801da8cf392 in term 1.
I20260808 18:20:13.728478 16201 raft_consensus.cc:2781] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:13.728682 16201 raft_consensus.cc:686] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [term 1 LEADER]: Becoming Leader. State: Replica: 57fc24c7d47246d381985801da8cf392, State: Running, Role: LEADER
I20260808 18:20:13.728986 16201 consensus_queue.cc:237] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } }
I20260808 18:20:13.732021 15366 catalog_manager.cc:5495] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57fc24c7d47246d381985801da8cf392 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "57fc24c7d47246d381985801da8cf392" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57fc24c7d47246d381985801da8cf392" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38743 } health_report { overall_health: HEALTHY } } }
I20260808 18:20:13.780463 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.776546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.27.18.193:38743)
0808 18:20:13.776709 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:13.776714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:13.776730 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:20:13.776817 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:13.776820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:13.776886 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:13.777027 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:20:13.777235 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.777988 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.778814 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:13.779138 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:13.779243 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:13.779557 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:20:13.780078 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:20:13.780204 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:13.780256 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:20:13.780315 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 18:20:13.786834 15804 raft_consensus.cc:1261] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 1 FOLLOWER]: Refusing update from remote peer 57fc24c7d47246d381985801da8cf392: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:13.786878 16029 raft_consensus.cc:1261] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 1 FOLLOWER]: Refusing update from remote peer 57fc24c7d47246d381985801da8cf392: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:13.787601 16201 consensus_queue.cc:1035] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae4c87ea29e14914a2fa6f64aeaa9646" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:13.787981 16207 consensus_queue.cc:1035] T ebf75243462f4012a10c2d478cefdb86 P 57fc24c7d47246d381985801da8cf392 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0af7c14bd34312beac2f75633d8a0e" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:13.796105 16224 mvcc.cc:205] Tried to move back new op lower bound from 7316329323663310848 to 7316329323436503040. Current Snapshot: MvccSnapshot[applied={T|T < 7316329323663310848}]
I20260808 18:20:13.797510 16226 mvcc.cc:205] Tried to move back new op lower bound from 7316329323663310848 to 7316329323436503040. Current Snapshot: MvccSnapshot[applied={T|T < 7316329323663310848}]
I20260808 18:20:13.799906 16225 mvcc.cc:205] Tried to move back new op lower bound from 7316329323663310848 to 7316329323436503040. Current Snapshot: MvccSnapshot[applied={T|T < 7316329323663310848}]
I20260808 18:20:13.807984 15366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:13.808149 15366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:13.810527 15366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e14c0ee5a5643138d665c3d241ec485: Sending DeleteTablet for 3 replicas of tablet ebf75243462f4012a10c2d478cefdb86
I20260808 18:20:13.811295 15797 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf75243462f4012a10c2d478cefdb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:13 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:52104
I20260808 18:20:13.811422 15988 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf75243462f4012a10c2d478cefdb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:13 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:33920
I20260808 18:20:13.811887 15558 tablet_service.cc:1514] Processing DeleteTablet for tablet ebf75243462f4012a10c2d478cefdb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:13 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55682
I20260808 18:20:13.812194 16236 tablet_replica.cc:330] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: stopping tablet replica
I20260808 18:20:13.812493 16236 raft_consensus.cc:2229] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:13.812783 16236 raft_consensus.cc:2258] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:13.813056 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 15430
I20260808 18:20:13.813133 16235 tablet_replica.cc:330] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: stopping tablet replica
I20260808 18:20:13.813779 16235 raft_consensus.cc:2229] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:13.813985 16235 raft_consensus.cc:2258] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:13.818118 16235 ts_tablet_manager.cc:1905] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:13.821029 16236 ts_tablet_manager.cc:1905] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:13.821784 16235 ts_tablet_manager.cc:1918] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:13.821902 16235 log.cc:1196] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/ebf75243462f4012a10c2d478cefdb86
I20260808 18:20:13.822291 16235 ts_tablet_manager.cc:1939] T ebf75243462f4012a10c2d478cefdb86 P ae4c87ea29e14914a2fa6f64aeaa9646: Deleting consensus metadata
I20260808 18:20:13.823190 16236 ts_tablet_manager.cc:1918] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:13.823300 16236 log.cc:1196] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/ebf75243462f4012a10c2d478cefdb86
I20260808 18:20:13.823563 15342 catalog_manager.cc:4841] TS ae4c87ea29e14914a2fa6f64aeaa9646 (127.27.18.194:40925): tablet ebf75243462f4012a10c2d478cefdb86 (table test-table [id=5d61d92b204448ae8e776db395454d62]) successfully deleted
I20260808 18:20:13.823717 16236 ts_tablet_manager.cc:1939] T ebf75243462f4012a10c2d478cefdb86 P 0b0af7c14bd34312beac2f75633d8a0e: Deleting consensus metadata
I20260808 18:20:13.824702 15343 catalog_manager.cc:4841] TS 0b0af7c14bd34312beac2f75633d8a0e (127.27.18.195:46203): tablet ebf75243462f4012a10c2d478cefdb86 (table test-table [id=5d61d92b204448ae8e776db395454d62]) successfully deleted
W20260808 18:20:13.827879 15342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:38743 (error 108)
I20260808 18:20:13.828068 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 15680
I20260808 18:20:13.831004 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:13.828734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38743 (local address 127.0.0.1:55690)
0808 18:20:13.830849 (+ 2115us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:13.830923 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:38743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2027}
W20260808 18:20:13.831166 15342 catalog_manager.cc:4584] TS 57fc24c7d47246d381985801da8cf392 (127.27.18.193:38743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebf75243462f4012a10c2d478cefdb86: Network error: Client connection negotiation failed: client connection to 127.27.18.193:38743: connect: Connection refused (error 111)
I20260808 18:20:13.840860 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 15906
I20260808 18:20:13.860229 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 15310
I20260808 18:20:13.875180 27723 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 18:20:14.220221 27723 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 18:20:14.220333 27723 mini_ranger.cc:66] Stopping Ranger...
I20260808 18:20:14.576118 27723 mini_ranger.cc:68] Stopped Ranger
2026-08-08 18:20:14.576 UTC [13806] LOG: received smart shutdown request
2026-08-08 18:20:14.580 UTC [13806] LOG: background worker "logical replication launcher" (PID 13823) exited with exit code 1
2026-08-08 18:20:14.580 UTC [13813] LOG: shutting down
2026-08-08 18:20:14.717 UTC [13806] LOG: database system is shut down
2026-08-08T18:20:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46282 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:20:14 dist-test-slave-x4qh krb5kdc[16254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:20:14 dist-test-slave-x4qh krb5kdc[16254](info): set up 2 sockets
Aug 08 18:20:14 dist-test-slave-x4qh krb5kdc[16254](info): commencing operation
krb5kdc: starting...
W20260808 18:20:15.997542 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:20:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:20:16.115078 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:43551
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:43551,127.27.18.253:38397,127.27.18.252:39023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:16.218302 16277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:16.218503 16277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:16.218554 16277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:16.222191 16277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:16.222239 16277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:16.222256 16277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:16.222271 16277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:16.222285 16277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:16.227135 16277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:43551,127.27.18.253:38397,127.27.18.252:39023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:43551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:16.228065 16277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:16.228984 16277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:16.236821 16286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:16.236975 16285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:16.237102 16277 server_base.cc:1034] running on GCE node
W20260808 18:20:16.236974 16289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:16.237730 16277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:16.238687 16277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:16.239861 16277 hybrid_clock.cc:648] HybridClock initialized: now 1786213216239852 us; error 38 us; skew 500 ppm
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:16.242432 16277 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:16.243173 16277 webserver.cc:460] Webserver started at http://127.27.18.254:41843/ using document root <none> and password file <none>
I20260808 18:20:16.243712 16277 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:16.243794 16277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:16.244001 16277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:16.245911 16277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "c8c28b0e89d44b658c4457e6e389aca8"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "cdf6b4195fe6dd88c58f56358fa2862c"
server_key_iv: "a51a109397199bf70a42aaf7ab5af172"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.246461 16277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "c8c28b0e89d44b658c4457e6e389aca8"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "cdf6b4195fe6dd88c58f56358fa2862c"
server_key_iv: "a51a109397199bf70a42aaf7ab5af172"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.249847 16277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:20:16.252184 16300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:16.253271 16277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:16.253417 16277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "c8c28b0e89d44b658c4457e6e389aca8"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "cdf6b4195fe6dd88c58f56358fa2862c"
server_key_iv: "a51a109397199bf70a42aaf7ab5af172"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.253527 16277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:16.271858 16277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:16.274780 16277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:16.274925 16277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:16.281774 16277 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:43551
I20260808 18:20:16.281785 16384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:43551 every 8 connection(s)
I20260808 18:20:16.282603 16277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:16.285240 16386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:16.288758 16386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:20:16.291118 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 16277
I20260808 18:20:16.291244 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:20:16.291545 27723 external_mini_cluster.cc:1428] Setting key e7dc9e3375ccf7a2efa57c1fa588ac06
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.253@KRBTEST.COM" created.
I20260808 18:20:16.303835 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.290653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.27.18.254:43551)
0808 18:20:16.291051 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.291063 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.291721 (+ 658us) server_negotiation.cc:407] Connection header received
0808 18:20:16.293027 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.293032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.293201 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.293368 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.294232 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.295502 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.296433 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.296795 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.296955 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.299461 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.299466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.299474 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.299493 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.301103 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.301521 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.301525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.301526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.301622 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.301932 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.301934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.301935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.302263 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.302374 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.302716 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.303047 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":276,"spinlock_wait_cycles":19200,"thread_start_us":155,"threads_started":1}
I20260808 18:20:16.303836 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.290653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43551 (local address 127.0.0.1:52860)
0808 18:20:16.291568 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.291616 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.292737 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.293427 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.293435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.293846 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.294080 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.294092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.295611 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.295614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.296316 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.296325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.296943 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.296947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.297054 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.297649 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.297664 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.299278 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.301203 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.301210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.301218 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.301427 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.301728 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.301730 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.301731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.301845 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.302370 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.302374 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.302489 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.302621 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.302974 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260808 18:20:16.306318 16386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:20:16.307024 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.306799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58378)
0808 18:20:16.306910 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.306974 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 18:20:16.307211 16302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
I20260808 18:20:16.308012 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.307791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58382)
0808 18:20:16.307897 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.307947 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:20:16.308167 16386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:38397: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab.
Entry for principal kudu/127.27.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab.
I20260808 18:20:16.318147 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:38397
--webserver_interface=127.27.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab
--principal=kudu/127.27.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:43551,127.27.18.253:38397,127.27.18.252:39023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:20:16.357401 16386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } attempt: 1
I20260808 18:20:16.358260 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.357971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58398)
0808 18:20:16.358119 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.358187 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 18:20:16.359134 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.358846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58408)
0808 18:20:16.359003 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.359047 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 18:20:16.359354 16386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:38397: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
W20260808 18:20:16.423055 16393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:16.423262 16393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:16.423297 16393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:16.427022 16393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:16.427074 16393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:16.427093 16393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:16.427109 16393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:16.427122 16393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:16.427572 16386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } attempt: 2
I20260808 18:20:16.428378 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.428097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58418)
0808 18:20:16.428249 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.428308 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 18:20:16.429116 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.428911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58424)
0808 18:20:16.429013 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.429063 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 18:20:16.429225 16386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:38397: Network error: Client connection negotiation failed: client connection to 127.27.18.253:38397: connect: Connection refused (error 111)
I20260808 18:20:16.432178 16393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:43551,127.27.18.253:38397,127.27.18.252:39023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab
--principal=kudu/127.27.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:16.433147 16393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:16.434032 16393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:16.441885 16399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:16.441864 16407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:16.441879 16411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:16.442092 16393 server_base.cc:1034] running on GCE node
I20260808 18:20:16.442582 16393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:16.443605 16393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:16.444782 16393 hybrid_clock.cc:648] HybridClock initialized: now 1786213216444765 us; error 29 us; skew 500 ppm
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:16.447288 16393 init.cc:377] Logged in from keytab as kudu/127.27.18.253@KRBTEST.COM (short username kudu)
I20260808 18:20:16.448051 16393 webserver.cc:460] Webserver started at http://127.27.18.253:46161/ using document root <none> and password file <none>
I20260808 18:20:16.448596 16393 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:16.448688 16393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:16.448915 16393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:16.450723 16393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data/instance:
uuid: "9fbf35cb060e4396b6283687a776e93c"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "748b08c75df77e7b0be879fb9158806a"
server_key_iv: "a2e8d06789478bdce0e93eb25e1b1476"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.451241 16393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal/instance:
uuid: "9fbf35cb060e4396b6283687a776e93c"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "748b08c75df77e7b0be879fb9158806a"
server_key_iv: "a2e8d06789478bdce0e93eb25e1b1476"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.454602 16393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 18:20:16.456934 16422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:16.458099 16393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:20:16.458235 16393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal
uuid: "9fbf35cb060e4396b6283687a776e93c"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "748b08c75df77e7b0be879fb9158806a"
server_key_iv: "a2e8d06789478bdce0e93eb25e1b1476"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.458341 16393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:16.474303 16393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:16.476807 16393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:16.476995 16393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:16.483729 16393 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.253:38397
I20260808 18:20:16.483754 16501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.253:38397 every 8 connection(s)
I20260808 18:20:16.484591 16393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
I20260808 18:20:16.487071 16503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:16.490375 16503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:20:16.494032 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 16393
I20260808 18:20:16.494185 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-1/wal/instance
I20260808 18:20:16.494464 27723 external_mini_cluster.cc:1428] Setting key 5ea122ed77dd545121c253d1bb72aa40
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:16.503764 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.492034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.27.18.254:43551)
0808 18:20:16.492154 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.492158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.492821 (+ 663us) server_negotiation.cc:407] Connection header received
0808 18:20:16.493562 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.493565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.493613 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.493684 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.494837 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.495311 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.496447 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.496831 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.496931 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.500173 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.500176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.500178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.500199 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.501651 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.502234 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.502236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.502237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.502314 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.502726 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.502728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.502729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.502881 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.502950 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.503563 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.503656 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.27.18.252@KRBTEST.COM; defaulting to no policy
I20260808 18:20:16.504798 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.492277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43551 (local address 127.0.0.1:52872)
0808 18:20:16.492668 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.492703 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.493370 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.493847 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.493853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.494174 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.494697 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.494711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.495454 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.495459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.496294 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.496316 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.497135 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.497140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.497392 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.498006 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.498029 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.499989 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.501766 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.501774 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.501787 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.502132 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.502423 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.502426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.502429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.502612 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.502987 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.502994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.503279 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.503517 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.503858 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":97,"threads_started":1}
I20260808 18:20:16.506171 16503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.252@KRBTEST.COM" created.
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:20:16.515388 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.506604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58436)
0808 18:20:16.506711 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.506727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.506799 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.507459 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.507462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.507583 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.507660 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.507664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.509060 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.509065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.509958 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.509972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.510510 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.510513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.510581 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.511035 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.511046 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.512178 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.513921 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.513928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.513930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.514124 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.514405 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.514409 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.514411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.514500 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.514895 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.514897 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.514972 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.515082 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.515211 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 18:20:16.515621 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.506727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.27.18.253:38397)
0808 18:20:16.507056 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.507064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.507081 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:20:16.507113 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.507117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.507306 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.507452 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.507768 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.508927 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.510085 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.510413 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.510491 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.512296 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.512316 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.512325 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.512343 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.513812 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.514228 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.514239 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.514241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.514314 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.514586 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.514588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.514590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.514789 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.514914 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.515173 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.515296 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":171,"threads_started":1}
I20260808 18:20:16.517263 16503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:20:16.517932 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.517648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38870)
0808 18:20:16.517795 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.517876 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 18:20:16.518091 16424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
I20260808 18:20:16.518932 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.518645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38884)
0808 18:20:16.518780 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.518846 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 18:20:16.519062 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:39023: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
I20260808 18:20:16.520368 16386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab.
Entry for principal kudu/127.27.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab.
I20260808 18:20:16.522110 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:39023
--webserver_interface=127.27.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab
--principal=kudu/127.27.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:43551,127.27.18.253:38397,127.27.18.252:39023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:20:16.528689 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.520789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58450)
0808 18:20:16.520913 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.520928 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.521028 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.521253 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.521259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.521466 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.521588 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.521593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.522491 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.522494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.522949 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.522955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.523596 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.523602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.523722 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.524113 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.524128 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.525394 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.527083 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.527089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.527092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.527384 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.527753 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.527756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.527758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.527872 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.528331 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.528333 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.528408 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.528485 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.528592 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 18:20:16.528924 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.520858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.27.18.253:38397)
0808 18:20:16.520965 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.520968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.520979 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:20:16.521116 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.521118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.521150 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.521203 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.521703 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.522389 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.523055 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.523454 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.523566 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.525522 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.525527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.525534 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.525554 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.526979 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.527542 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.527547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.527550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.527639 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.527972 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.527977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.527981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.528220 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.528334 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.528588 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.528709 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:20:16.529966 16386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:20:16.530642 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.530375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38896)
0808 18:20:16.530504 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.530564 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 18:20:16.531340 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.531050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38902)
0808 18:20:16.531214 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.531273 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 18:20:16.531497 16386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:39023: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
I20260808 18:20:16.562670 16386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } attempt: 1
I20260808 18:20:16.563517 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.563105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38916)
0808 18:20:16.563282 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.563362 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 18:20:16.564337 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.564071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38922)
0808 18:20:16.564200 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.564262 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:20:16.564522 16386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:39023: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
I20260808 18:20:16.568276 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } attempt: 1
I20260808 18:20:16.568914 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.568628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38932)
0808 18:20:16.568764 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.568855 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 18:20:16.569617 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.569432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38942)
0808 18:20:16.569528 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.569570 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:20:16.569784 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:39023: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
W20260808 18:20:16.629089 16510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:16.629334 16510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:16.629379 16510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:16.633204 16510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:16.633252 16510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:16.633270 16510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:16.633314 16510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:16.633343 16510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:16.637938 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } attempt: 2
I20260808 18:20:16.638072 16510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:43551,127.27.18.253:38397,127.27.18.252:39023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab
--principal=kudu/127.27.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:39023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:16.638742 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.638477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38956)
0808 18:20:16.638610 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.638672 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 18:20:16.639101 16510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:16.639506 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.639248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38958)
0808 18:20:16.639346 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.639439 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 18:20:16.639639 16503 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:39023: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
I20260808 18:20:16.639685 16386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } attempt: 2
I20260808 18:20:16.640285 16510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:20:16.640329 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.640045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38974)
0808 18:20:16.640188 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.640248 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 18:20:16.640978 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.640767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:38986)
0808 18:20:16.640875 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.640925 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:20:16.641111 16386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:39023: Network error: Client connection negotiation failed: client connection to 127.27.18.252:39023: connect: Connection refused (error 111)
W20260808 18:20:16.648540 16517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:16.648528 16518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:16.648519 16520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:16.648599 16510 server_base.cc:1034] running on GCE node
I20260808 18:20:16.649250 16510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:16.650241 16510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:16.651429 16510 hybrid_clock.cc:648] HybridClock initialized: now 1786213216651415 us; error 22 us; skew 500 ppm
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:16.653990 16510 init.cc:377] Logged in from keytab as kudu/127.27.18.252@KRBTEST.COM (short username kudu)
I20260808 18:20:16.654866 16510 webserver.cc:460] Webserver started at http://127.27.18.252:45131/ using document root <none> and password file <none>
I20260808 18:20:16.655447 16510 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:16.655521 16510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:16.655740 16510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:16.657719 16510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data/instance:
uuid: "7ad603490297424baa0ca45e03f85877"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "b397d1e3bec126560a991c80785911e2"
server_key_iv: "dbb02c61c2762d2d7075957a9e2eb1a3"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.658255 16510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal/instance:
uuid: "7ad603490297424baa0ca45e03f85877"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "b397d1e3bec126560a991c80785911e2"
server_key_iv: "dbb02c61c2762d2d7075957a9e2eb1a3"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.661708 16510 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:20:16.663954 16533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:16.665081 16510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:16.665198 16510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal
uuid: "7ad603490297424baa0ca45e03f85877"
format_stamp: "Formatted at 2026-08-08 18:20:16 on dist-test-slave-x4qh"
server_key: "b397d1e3bec126560a991c80785911e2"
server_key_iv: "dbb02c61c2762d2d7075957a9e2eb1a3"
server_key_version: "encryptionkey@0"
I20260808 18:20:16.665292 16510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:16.683671 16510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:16.686178 16510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:16.686374 16510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:16.693250 16510 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.252:39023
I20260808 18:20:16.693279 16611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.252:39023 every 8 connection(s)
I20260808 18:20:16.694051 16510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
I20260808 18:20:16.696576 16613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:16.698189 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 16510
I20260808 18:20:16.698340 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786213058405439-27723-0/minicluster-data/master-2/wal/instance
I20260808 18:20:16.698614 27723 external_mini_cluster.cc:1428] Setting key 99bdfbc994eb0c7c20b336aa52733bc8
I20260808 18:20:16.700695 16613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:16.711998 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.702866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.27.18.254:43551)
0808 18:20:16.703009 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.703014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.703230 (+ 216us) server_negotiation.cc:407] Connection header received
0808 18:20:16.703362 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.703365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.703440 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.703522 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.704132 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.704646 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.705573 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.705794 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.705917 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.708504 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.708507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.708509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.708532 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.710205 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.710645 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.710653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.710654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.710723 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.711024 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.711027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.711029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.711194 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.711302 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.711728 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.711847 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 18:20:16.712489 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.699955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.27.18.252:39023)
0808 18:20:16.700387 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.700406 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.700446 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:20:16.701153 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.701177 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.701561 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.701893 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.702560 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.703839 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.704706 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.705204 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.705380 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.708091 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.708097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.708105 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.708127 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.709803 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.710235 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.710245 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.710249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.710350 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.710621 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.710624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.710626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.710970 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.711175 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.711452 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.711738 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":299,"thread_start_us":115,"threads_started":1}
I20260808 18:20:16.712500 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.702781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43551 (local address 127.0.0.1:52882)
0808 18:20:16.703105 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.703150 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.703262 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.703576 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.703581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.703892 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.704004 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.704010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.704775 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.704785 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.705447 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.705456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.705955 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.705959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.706062 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.706556 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.706571 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.708361 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.710320 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.710328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.710333 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.710542 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.710810 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.710816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.710819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.710925 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.711309 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.711317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.711476 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.711637 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.711792 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":28,"thread_start_us":101,"threads_started":1}
I20260808 18:20:16.713915 16613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:16.723203 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.714335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:38397 (local address 127.0.0.1:58452)
0808 18:20:16.714428 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.714441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.714512 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.714945 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.714948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.715105 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.715167 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.715172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.715919 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.715924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.716429 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.716436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.717970 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.717974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.718050 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.718562 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.718572 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.719786 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.721817 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.721819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.721820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.722000 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.722264 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.722267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.722269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.722355 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.722723 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.722726 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.722820 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.722940 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.723059 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 18:20:16.723268 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.714590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.27.18.253:38397)
0808 18:20:16.714701 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.714704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.714715 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:20:16.714777 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.714780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.714843 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.714929 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.715279 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.715821 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.717635 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.717859 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.717968 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.719922 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.719926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.719928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.719952 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.721695 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.722096 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.722099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.722100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.722184 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.722450 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.722455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.722457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.722631 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.722728 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.723010 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.723111 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 18:20:16.723830 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.714898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.27.18.254:43551)
0808 18:20:16.715007 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.715010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.715023 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:20:16.715143 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.715146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.715182 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.715268 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.715878 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.716330 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.717516 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.717811 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.717906 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.720501 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.720506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.720509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.720537 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.722088 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.722464 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.722469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.722470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.722571 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.722833 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.722837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.722838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.723030 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.723236 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.723597 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.723713 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:20:16.724221 16613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:20:16.730795 16503 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } attempt: 3
I20260808 18:20:16.732486 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.724710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:39008)
0808 18:20:16.724794 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.724808 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.724877 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.725086 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.725089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.725233 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.725288 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.725292 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.726396 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.726399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.726854 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.726861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.727293 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.727296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.727428 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.727861 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.727879 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.729100 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.731018 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.731021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.731023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.731257 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.731571 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.731575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.731577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.731653 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.732034 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.732036 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.732119 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.732218 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.732338 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":20}
I20260808 18:20:16.732620 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.724764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.27.18.252:39023)
0808 18:20:16.724871 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.724875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.724889 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:20:16.724952 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.724955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.724999 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.725074 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.725396 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.726275 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.726963 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.727188 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.727291 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.729243 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.729246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.729247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.729267 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.730903 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.731399 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.731405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.731407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.731499 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.731743 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.731745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.731747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.731912 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.732000 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.732303 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.732405 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":44}
I20260808 18:20:16.734364 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.725047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.27.18.253:38397)
0808 18:20:16.725154 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.725163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.725172 (+ 9us) server_negotiation.cc:407] Connection header received
0808 18:20:16.725198 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.725200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.725228 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.725321 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.725885 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.726362 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.727072 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.727281 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.727406 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.729897 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.729901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.729904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.729929 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.731513 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.731962 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.731965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.731966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.732047 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.732325 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.732327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.732329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.732513 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.732735 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.734142 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.734235 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:20:16.736523 16613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877: Bootstrap starting.
I20260808 18:20:16.737640 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:20:16.739270 16613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:16.739858 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.731175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:39018)
0808 18:20:16.731342 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.731357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.731442 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.731963 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.731968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.732130 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.732232 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.732238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.733305 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.733308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.733867 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.733876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.734364 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.734368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.734454 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.734936 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.734947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.736209 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.738208 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.738211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.738213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.738448 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.738784 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.738786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.738788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.738888 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.739410 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.739414 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.739511 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.739606 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.739718 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 18:20:16.740187 16613 log.cc:824] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:16.740247 16386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } attempt: 3
I20260808 18:20:16.740434 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.731266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.27.18.252:39023)
0808 18:20:16.731742 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.731757 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.731769 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:20:16.731801 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.731803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.731841 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.731926 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.732346 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.733185 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.733967 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.734247 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.734349 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.736343 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.736346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.736349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.736371 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.738089 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.738590 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.738594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.738595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.738682 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.739000 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.739003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.739005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.739249 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.740111 (+ 862us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.740182 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.740289 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":130,"threads_started":1}
I20260808 18:20:16.743211 16503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: Bootstrap starting.
Aug 08 18:20:16 dist-test-slave-x4qh krb5kdc[16254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213216, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:20:16.745764 16503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:16.746613 16503 log.cc:824] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:16.748912 16613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877: No bootstrap required, opened a new log
I20260808 18:20:16.750278 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.740590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:39023 (local address 127.0.0.1:39022)
0808 18:20:16.740681 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:16.740697 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:16.740779 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:16.741253 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:16.741259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:16.741454 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:16.741568 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.741577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.742480 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.742483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.743288 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:16.743313 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.744009 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.744012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:16.744095 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.744529 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:16.744549 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:16.745983 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:16.747847 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.747850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.747852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.748135 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.748487 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:16.748493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:16.748495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:16.748588 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:16.749744 (+ 1156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:16.749747 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:16.749852 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 18:20:16.749984 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 18:20:16.750110 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 18:20:16.751443 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.740703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.27.18.252:39023)
0808 18:20:16.740910 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.740914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.740925 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:20:16.740958 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.740960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.740998 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.741067 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.741710 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.742345 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.743488 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.743816 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.743907 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.746120 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.746123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.746125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.746148 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.747729 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.748270 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.748274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.748276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.748373 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.748696 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.748702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.748706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.749275 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.749702 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.750796 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.751241 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":325,"server-negotiator.queue_time_us":138}
I20260808 18:20:16.752487 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.740832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58470 (local address 127.27.18.253:38397)
0808 18:20:16.741197 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.741200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.742306 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 18:20:16.742475 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.742478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.742516 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.742561 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.743485 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.744170 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.746561 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.746927 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.747004 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.748593 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.748596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.748598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.748623 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.750257 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.750773 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.750776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.750778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.750872 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.751198 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.751204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.751207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.751550 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.751661 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.752222 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.752333 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296}
I20260808 18:20:16.752990 16613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.753219 16613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:16.753273 16613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad603490297424baa0ca45e03f85877, State: Initialized, Role: FOLLOWER
I20260808 18:20:16.753643 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.740179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.27.18.254:43551)
0808 18:20:16.740945 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.740948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.740959 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:20:16.740987 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.740990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.741026 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.741093 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.741986 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.742658 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.743764 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.744066 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.744419 (+ 353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.745863 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.745869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.745872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.745900 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.751135 (+ 5235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.752034 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.752037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.752039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.752125 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.752451 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.752454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.752455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.752839 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.752951 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.753421 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.753521 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702}
I20260808 18:20:16.753813 16613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.754112 16386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: Bootstrap starting.
I20260808 18:20:16.754316 16636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.754464 16636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:20:16.755103 16613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:16.756631 16386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:16.757586 16386 log.cc:824] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:16.759591 16503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: No bootstrap required, opened a new log
I20260808 18:20:16.759790 16386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: No bootstrap required, opened a new log
W20260808 18:20:16.760918 16659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:20:16.760995 16659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:20:16.761332 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:16.741341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.27.18.252:39023)
0808 18:20:16.742108 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:16.742116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:16.746413 (+ 4297us) server_negotiation.cc:407] Connection header received
0808 18:20:16.746902 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:16.746905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:16.746976 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:16.748051 (+ 1075us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:16.748881 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.749581 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.750674 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:16.751006 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:16.751077 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:16.757273 (+ 6196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:16.757277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:16.757280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:16.757305 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:16.759204 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.759782 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.759785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.759787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.759904 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:16.760262 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:16.760265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:16.760266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:16.760481 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:16.760589 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:16.760949 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 18:20:16.761157 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":706,"thread_start_us":89,"threads_started":1}
I20260808 18:20:16.763693 16386 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.763904 16386 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:16.763965 16386 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8c28b0e89d44b658c4457e6e389aca8, State: Initialized, Role: FOLLOWER
I20260808 18:20:16.764458 16386 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.762588 16510 master_runner.cc:186] Error getting master registration for 127.27.18.254:43551: OK, instance_id { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" instance_seqno: 1786213216279856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:20:16.765681 16510 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:20:16.765703 16386 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:16.766076 16503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.766258 16503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:16.766316 16503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbf35cb060e4396b6283687a776e93c, State: Initialized, Role: FOLLOWER
I20260808 18:20:16.766796 16503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.767257 16671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.767477 16671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:20:16.768038 16503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:16.768303 16661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.768425 16661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 18:20:16.772222 16678 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:20:16.772323 16678 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:20:16.770161 16277 master_runner.cc:186] Error getting master registration for 127.27.18.253:38397: OK, instance_id { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" instance_seqno: 1786213216481856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:20:16.773924 16277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:20:16.775770 16393 master_runner.cc:263] No leader master found from master 9fbf35cb060e4396b6283687a776e93c
I20260808 18:20:16.775856 16393 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 18:20:16.776428 16310 tablet.cc:2369] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:20:16.778009 16695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:20:16.778088 16695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:20:16.814030 16636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:16.814200 16636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.815143 16636 leader_election.cc:288] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8c28b0e89d44b658c4457e6e389aca8 (127.27.18.254:43551), 9fbf35cb060e4396b6283687a776e93c (127.27.18.253:38397)
I20260808 18:20:16.815714 16460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ad603490297424baa0ca45e03f85877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbf35cb060e4396b6283687a776e93c" is_pre_election: true
I20260808 18:20:16.815932 16341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ad603490297424baa0ca45e03f85877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8c28b0e89d44b658c4457e6e389aca8" is_pre_election: true
I20260808 18:20:16.816082 16460 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ad603490297424baa0ca45e03f85877 in term 0.
I20260808 18:20:16.816233 16341 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ad603490297424baa0ca45e03f85877 in term 0.
I20260808 18:20:16.816493 16535 leader_election.cc:302] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ad603490297424baa0ca45e03f85877, 9fbf35cb060e4396b6283687a776e93c; no voters:
I20260808 18:20:16.817660 16636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:20:16.817749 16636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:20:16.817813 16636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:16.819054 16636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.819659 16460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ad603490297424baa0ca45e03f85877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbf35cb060e4396b6283687a776e93c"
I20260808 18:20:16.819737 16341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ad603490297424baa0ca45e03f85877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8c28b0e89d44b658c4457e6e389aca8"
I20260808 18:20:16.819819 16636 leader_election.cc:288] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [CANDIDATE]: Term 1 election: Requested vote from peers c8c28b0e89d44b658c4457e6e389aca8 (127.27.18.254:43551), 9fbf35cb060e4396b6283687a776e93c (127.27.18.253:38397)
I20260808 18:20:16.819784 16460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:16.819859 16341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:16.821038 16460 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ad603490297424baa0ca45e03f85877 in term 1.
I20260808 18:20:16.821137 16341 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ad603490297424baa0ca45e03f85877 in term 1.
I20260808 18:20:16.821322 16535 leader_election.cc:302] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ad603490297424baa0ca45e03f85877, 9fbf35cb060e4396b6283687a776e93c; no voters:
I20260808 18:20:16.821493 16636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:16.821687 16636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [term 1 LEADER]: Becoming Leader. State: Replica: 7ad603490297424baa0ca45e03f85877, State: Running, Role: LEADER
I20260808 18:20:16.821981 16636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } }
I20260808 18:20:16.823546 16636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ad603490297424baa0ca45e03f85877. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.823649 16636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:16.830050 16703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:16.830924 16703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:16.836834 16460 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [term 1 FOLLOWER]: Refusing update from remote peer 7ad603490297424baa0ca45e03f85877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:16.837435 16698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:16.842614 16671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ad603490297424baa0ca45e03f85877. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.842723 16671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:20:16.844094 16341 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [term 1 FOLLOWER]: Refusing update from remote peer 7ad603490297424baa0ca45e03f85877: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:16.844652 16698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:16.848253 16661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ad603490297424baa0ca45e03f85877. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.848371 16661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:20:16.850802 16698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.850922 16698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:16.851926 16661 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.852036 16661 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:20:16.852404 16636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.852517 16636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:16.853551 16703 catalog_manager.cc:1347] Generated new cluster ID: f567606b9de44a3aa651c9ab012d8b99
I20260808 18:20:16.853634 16703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:16.857131 16671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ad603490297424baa0ca45e03f85877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8c28b0e89d44b658c4457e6e389aca8" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 43551 } } peers { permanent_uuid: "9fbf35cb060e4396b6283687a776e93c" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 38397 } } peers { permanent_uuid: "7ad603490297424baa0ca45e03f85877" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 39023 } } }
I20260808 18:20:16.857280 16671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:20:16.858392 16710 mvcc.cc:205] Tried to move back new op lower bound from 7316329336156909568 to 7316329336105480192. Current Snapshot: MvccSnapshot[applied={T|T < 7316329336156909568}]
I20260808 18:20:16.861268 16705 mvcc.cc:205] Tried to move back new op lower bound from 7316329336156909568 to 7316329336105480192. Current Snapshot: MvccSnapshot[applied={T|T < 7316329336156909568 or (T in {7316329336156909568})}]
I20260808 18:20:16.864240 16706 mvcc.cc:205] Tried to move back new op lower bound from 7316329336156909568 to 7316329336105480192. Current Snapshot: MvccSnapshot[applied={T|T < 7316329336156909568 or (T in {7316329336156909568})}]
I20260808 18:20:16.869518 16703 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:20:16.870721 16703 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:16.896339 16703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877: Generated new TSK 0
I20260808 18:20:16.897137 16703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:20:17.773574 16678 catalog_manager.cc:1259] Loaded cluster ID: f567606b9de44a3aa651c9ab012d8b99
I20260808 18:20:17.773676 16678 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: loading cluster ID for follower catalog manager: success
I20260808 18:20:17.775568 16678 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: acquiring CA information for follower catalog manager: success
I20260808 18:20:17.776681 16678 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c8c28b0e89d44b658c4457e6e389aca8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:20:17.779132 16695 catalog_manager.cc:1259] Loaded cluster ID: f567606b9de44a3aa651c9ab012d8b99
I20260808 18:20:17.779191 16695 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: loading cluster ID for follower catalog manager: success
I20260808 18:20:17.780709 16695 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: acquiring CA information for follower catalog manager: success
I20260808 18:20:17.781661 16695 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9fbf35cb060e4396b6283687a776e93c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:20:17.966544 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 16277
I20260808 18:20:17.978818 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 16393
I20260808 18:20:17.986795 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:17.986417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43551 (local address 127.0.0.1:52902)
0808 18:20:17.986636 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:17.986704 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
W20260808 18:20:17.986999 16536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.254:43551: connect: Connection refused (error 111)
I20260808 18:20:17.987871 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:17.987661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:43551 (local address 127.0.0.1:52906)
0808 18:20:17.987766 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:17.987818 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:43551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 18:20:17.987998 16536 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7ad603490297424baa0ca45e03f85877 -> Peer c8c28b0e89d44b658c4457e6e389aca8 (127.27.18.254:43551): Couldn't send request to peer c8c28b0e89d44b658c4457e6e389aca8. Status: Network error: Client connection negotiation failed: client connection to 127.27.18.254:43551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 18:20:17.989943 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 16510
2026-08-08T18:20:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3162 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:20:18 dist-test-slave-x4qh krb5kdc[16736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:20:18 dist-test-slave-x4qh krb5kdc[16736](info): set up 2 sockets
Aug 08 18:20:18 dist-test-slave-x4qh krb5kdc[16736](info): commencing operation
krb5kdc: starting...
W20260808 18:20:19.175554 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:20:19 dist-test-slave-x4qh krb5kdc[16736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:20:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:20:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:20:19.288369 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41447
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:41447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:19.393181 16757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:19.393410 16757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:19.393448 16757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:19.397248 16757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:19.397297 16757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:19.397315 16757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:19.397330 16757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:19.397346 16757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:19.402271 16757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:41447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:19.403183 16757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:19.404196 16757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:19.412061 16764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:19.412066 16769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:19.412066 16763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:19.412310 16757 server_base.cc:1034] running on GCE node
I20260808 18:20:19.412807 16757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:19.413834 16757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:19.415022 16757 hybrid_clock.cc:648] HybridClock initialized: now 1786213219415005 us; error 29 us; skew 500 ppm
Aug 08 18:20:19 dist-test-slave-x4qh krb5kdc[16736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213219, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:19.417673 16757 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:19.418526 16757 webserver.cc:460] Webserver started at http://127.27.18.254:38037/ using document root <none> and password file <none>
I20260808 18:20:19.419065 16757 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:19.419157 16757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:19.419435 16757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:19.421317 16757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "9100a7e288974b52a3be843ef097dc53"
format_stamp: "Formatted at 2026-08-08 18:20:19 on dist-test-slave-x4qh"
server_key: "9b6c57466bf64fa4920ece4a4398e45f"
server_key_iv: "e01764eb4acec1cfa89c3b13b769a1af"
server_key_version: "encryptionkey@0"
I20260808 18:20:19.421892 16757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "9100a7e288974b52a3be843ef097dc53"
format_stamp: "Formatted at 2026-08-08 18:20:19 on dist-test-slave-x4qh"
server_key: "9b6c57466bf64fa4920ece4a4398e45f"
server_key_iv: "e01764eb4acec1cfa89c3b13b769a1af"
server_key_version: "encryptionkey@0"
I20260808 18:20:19.425431 16757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:20:19.427798 16778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:19.428923 16757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:19.429060 16757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "9100a7e288974b52a3be843ef097dc53"
format_stamp: "Formatted at 2026-08-08 18:20:19 on dist-test-slave-x4qh"
server_key: "9b6c57466bf64fa4920ece4a4398e45f"
server_key_iv: "e01764eb4acec1cfa89c3b13b769a1af"
server_key_version: "encryptionkey@0"
I20260808 18:20:19.429167 16757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:19.444938 16757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:19.448026 16757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:19.448217 16757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:19.454753 16757 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:41447
I20260808 18:20:19.454780 16872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:41447 every 8 connection(s)
I20260808 18:20:19.455608 16757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:19.458170 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:19.463248 16873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53: Bootstrap starting.
I20260808 18:20:19.464386 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 16757
I20260808 18:20:19.464533 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:20:19.464847 27723 external_mini_cluster.cc:1428] Setting key b1467d6c41dc658eb824e46069b2ce75
I20260808 18:20:19.466008 16873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:19.466780 16873 log.cc:824] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:19.468675 16873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53: No bootstrap required, opened a new log
I20260808 18:20:19.471877 16873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9100a7e288974b52a3be843ef097dc53" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41447 } }
I20260808 18:20:19.472100 16873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:19.472178 16873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9100a7e288974b52a3be843ef097dc53, State: Initialized, Role: FOLLOWER
I20260808 18:20:19.472580 16873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9100a7e288974b52a3be843ef097dc53" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41447 } }
I20260808 18:20:19.472725 16873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:19.472817 16873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:19.472934 16873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:20:19 dist-test-slave-x4qh krb5kdc[16736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:19.473855 16873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9100a7e288974b52a3be843ef097dc53" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41447 } }
I20260808 18:20:19.474174 16873 leader_election.cc:302] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9100a7e288974b52a3be843ef097dc53; no voters:
I20260808 18:20:19.474516 16873 leader_election.cc:288] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:20:19.474610 16882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:19.474825 16882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [term 1 LEADER]: Becoming Leader. State: Replica: 9100a7e288974b52a3be843ef097dc53, State: Running, Role: LEADER
I20260808 18:20:19.475214 16882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9100a7e288974b52a3be843ef097dc53" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41447 } }
I20260808 18:20:19.475847 16873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:19.477134 16883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9100a7e288974b52a3be843ef097dc53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9100a7e288974b52a3be843ef097dc53" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41447 } } }
I20260808 18:20:19.477273 16883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:19.477710 16885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9100a7e288974b52a3be843ef097dc53. Latest consensus state: current_term: 1 leader_uuid: "9100a7e288974b52a3be843ef097dc53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9100a7e288974b52a3be843ef097dc53" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41447 } } }
I20260808 18:20:19.478762 16885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53 [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:19.480074 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:19.465881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.27.18.254:41447)
0808 18:20:19.466344 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:19.466359 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:19.466393 (+ 34us) server_negotiation.cc:407] Connection header received
0808 18:20:19.467084 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:19.467102 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:19.467548 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:19.467903 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:19.468730 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:19.469735 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:19.470584 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:19.471049 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:19.471218 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:19.473911 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:19.473917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:19.473939 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:19.473964 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:19.476162 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:19.476748 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:19.476757 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:19.476767 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:19.476924 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:19.477267 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:19.477271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:19.477273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:19.477855 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:19.478053 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:19.478845 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 18:20:19.479126 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":136,"threads_started":1}
W20260808 18:20:19.481222 16906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:20:19.481302 16906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:20:19.481390 16897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:19.482084 16897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:19.483570 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:20:19.488180 16897 catalog_manager.cc:1347] Generated new cluster ID: f8e66acebb3e4d618b41c50d5423baa6
I20260808 18:20:19.488246 16897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:19.512275 16897 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:20:19.513478 16897 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:19.520506 16897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9100a7e288974b52a3be843ef097dc53: Generated new TSK 0
I20260808 18:20:19.521224 16897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:20:19.539038 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 16757
2026-08-08T18:20:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1548 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:20:19 dist-test-slave-x4qh krb5kdc[16915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:20:19 dist-test-slave-x4qh krb5kdc[16915](info): set up 2 sockets
Aug 08 18:20:19 dist-test-slave-x4qh krb5kdc[16915](info): commencing operation
krb5kdc: starting...
W20260808 18:20:20.723853 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:20:20 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:20:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:20:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:20:20.838119 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38431
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:38431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:20.943939 16931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:20.944205 16931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:20.944278 16931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:20.948446 16931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:20.948529 16931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:20.948565 16931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:20.948622 16931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:20.948666 16931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:20.953244 16931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:38431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:20.954123 16931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:20.955018 16931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:20.963140 16936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:20.963135 16942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:20.963112 16939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:20.963536 16931 server_base.cc:1034] running on GCE node
I20260808 18:20:20.964102 16931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:20.965067 16931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:20.966249 16931 hybrid_clock.cc:648] HybridClock initialized: now 1786213220966223 us; error 37 us; skew 500 ppm
Aug 08 18:20:20 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213220, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:20.968894 16931 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:20.969681 16931 webserver.cc:460] Webserver started at http://127.27.18.254:37009/ using document root <none> and password file <none>
I20260808 18:20:20.970198 16931 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:20.970285 16931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:20.970502 16931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:20.972383 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "5f8500f10896422995e47e9b17df89cc"
format_stamp: "Formatted at 2026-08-08 18:20:20 on dist-test-slave-x4qh"
server_key: "8ec1e9da1d3658f3171e81c82a1b2c72"
server_key_iv: "bdc384e1185efff034964b9e8bcc64f1"
server_key_version: "encryptionkey@0"
I20260808 18:20:20.972923 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "5f8500f10896422995e47e9b17df89cc"
format_stamp: "Formatted at 2026-08-08 18:20:20 on dist-test-slave-x4qh"
server_key: "8ec1e9da1d3658f3171e81c82a1b2c72"
server_key_iv: "bdc384e1185efff034964b9e8bcc64f1"
server_key_version: "encryptionkey@0"
I20260808 18:20:20.976336 16931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:20:20.978518 16953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:20.979676 16931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:20.979817 16931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5f8500f10896422995e47e9b17df89cc"
format_stamp: "Formatted at 2026-08-08 18:20:20 on dist-test-slave-x4qh"
server_key: "8ec1e9da1d3658f3171e81c82a1b2c72"
server_key_iv: "bdc384e1185efff034964b9e8bcc64f1"
server_key_version: "encryptionkey@0"
I20260808 18:20:20.979913 16931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:21.027779 16931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:21.030332 16931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:21.030529 16931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:21.037261 16931 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:38431
I20260808 18:20:21.037751 17038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:38431 every 8 connection(s)
I20260808 18:20:21.038255 16931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:21.041020 17039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:21.044261 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 16931
I20260808 18:20:21.044438 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:20:21.044811 27723 external_mini_cluster.cc:1428] Setting key a4ebc3f0371c72d93d34abe200310658
I20260808 18:20:21.047365 17039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap starting.
I20260808 18:20:21.049944 17039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:21.050850 17039 log.cc:824] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:21.053488 17039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: No bootstrap required, opened a new log
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:21.056918 17039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:21.057154 17039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:21.057243 17039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f8500f10896422995e47e9b17df89cc, State: Initialized, Role: FOLLOWER
I20260808 18:20:21.057741 17039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:21.057891 17039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:21.057965 17039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:21.058053 17039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:21.058840 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.046247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.27.18.254:38431)
0808 18:20:21.046699 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.046718 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.046758 (+ 40us) server_negotiation.cc:407] Connection header received
0808 18:20:21.047360 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.047376 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.047774 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.048034 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:21.048969 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.049892 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.051052 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.051564 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.051693 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.054414 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:21.054419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:21.054430 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:21.054453 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:21.055985 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.056414 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.056418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.056422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.056540 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.056820 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.056826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.056829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.057197 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:21.057399 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.057696 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.057972 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":174,"threads_started":1}
I20260808 18:20:21.059016 17039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:21.059314 17039 leader_election.cc:302] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f8500f10896422995e47e9b17df89cc; no voters:
I20260808 18:20:21.059635 17039 leader_election.cc:288] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:20:21.059734 17048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:21.059945 17048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 LEADER]: Becoming Leader. State: Replica: 5f8500f10896422995e47e9b17df89cc, State: Running, Role: LEADER
I20260808 18:20:21.060257 17048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:21.060729 17039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:21.061417 17050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f8500f10896422995e47e9b17df89cc. Latest consensus state: current_term: 1 leader_uuid: "5f8500f10896422995e47e9b17df89cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } } }
I20260808 18:20:21.061561 17050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:21.061897 17049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f8500f10896422995e47e9b17df89cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } } }
I20260808 18:20:21.062014 17049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: This master's current role is: LEADER
W20260808 18:20:21.065891 17068 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:20:21.065966 17068 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:20:21.066056 17058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:21.066815 17058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:21.072190 17058 catalog_manager.cc:1347] Generated new cluster ID: d95c5f3493f6461b9736cc34ae49b00a
I20260808 18:20:21.072263 17058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:21.078770 17058 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:20:21.079561 17058 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:21.084941 17058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Generated new TSK 0
I20260808 18:20:21.085485 17058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:20:21.115479 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38431
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:21.244812 17075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:21.245012 17075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:21.245048 17075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:21.248880 17075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:21.248926 17075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:21.249020 17075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:20:21.253566 17075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:21.254407 17075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:21.255270 17075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:21.264437 17080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:21.264461 17082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:21.264503 17075 server_base.cc:1034] running on GCE node
W20260808 18:20:21.264456 17086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:21.265074 17075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:21.265604 17075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:21.266785 17075 hybrid_clock.cc:648] HybridClock initialized: now 1786213221266758 us; error 36 us; skew 500 ppm
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:21.269593 17075 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:20:21.270328 17075 webserver.cc:460] Webserver started at http://127.27.18.193:40737/ using document root <none> and password file <none>
I20260808 18:20:21.270787 17075 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:21.270853 17075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:21.271001 17075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:21.272927 17075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "59de6ece509c45e6ac435edc7178fc53"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e3e5d866a11fff2a0c8942e3d28f1924"
server_key_iv: "4ea87566d2d1330f52846673113f65b8"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.273485 17075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "59de6ece509c45e6ac435edc7178fc53"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e3e5d866a11fff2a0c8942e3d28f1924"
server_key_iv: "4ea87566d2d1330f52846673113f65b8"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.276950 17075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:20:21.279052 17095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.280169 17075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:21.280333 17075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "59de6ece509c45e6ac435edc7178fc53"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e3e5d866a11fff2a0c8942e3d28f1924"
server_key_iv: "4ea87566d2d1330f52846673113f65b8"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.280438 17075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:21.301403 17075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:21.303931 17075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:21.304122 17075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:21.304666 17075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:21.305464 17075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:21.305533 17075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.305601 17075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:21.305644 17075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.314218 17075 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:35539
I20260808 18:20:21.314244 17273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:35539 every 8 connection(s)
I20260808 18:20:21.314978 17075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:20:21.321902 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 17075
I20260808 18:20:21.322057 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:20:21.322391 27723 external_mini_cluster.cc:1428] Setting key c9cff24c8b35d50026a368c9f8a5330e
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:21.328718 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.316725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:57161 (local address 127.27.18.254:38431)
0808 18:20:21.316864 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.316868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.317604 (+ 736us) server_negotiation.cc:407] Connection header received
0808 18:20:21.318877 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.318880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.318936 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.319014 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:21.320095 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.320625 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.321716 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.321970 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.322082 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.324653 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:21.324656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:21.324659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:21.324686 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:21.326254 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.326924 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.326927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.326929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.327010 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.327456 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.327462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.327466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.327720 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:21.327827 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.328337 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.328485 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:20:21.329197 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.317030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:57161)
0808 18:20:21.317459 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.317497 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.318668 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.319166 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.319172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.319514 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:21.319946 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.319957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.320755 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.320758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.321583 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.321592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.322236 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.322242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.322448 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.323052 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:21.323067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:21.324497 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:21.326381 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:21.326389 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:21.326411 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:21.326791 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:21.327111 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:21.327115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:21.327116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:21.327293 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:21.327827 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:21.327832 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:21.328066 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.328225 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.328456 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":120,"threads_started":1}
I20260808 18:20:21.330353 17274 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38431
I20260808 18:20:21.330606 17274 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:21.331157 17274 heartbeater.cc:511] Master 127.27.18.254:38431 requested a full tablet report, sending...
I20260808 18:20:21.332574 16980 ts_manager.cc:194] Registered new tserver with Master: 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539)
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:20:21.333760 16980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:57161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:20:21.349392 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38431
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:21.454700 17285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:21.454911 17285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:21.454950 17285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:21.458581 17285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:21.458639 17285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:21.458693 17285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:20:21.463711 17285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:21.464663 17285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:21.465584 17285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:20:21.474975 17285 server_base.cc:1034] running on GCE node
W20260808 18:20:21.474963 17292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:21.474959 17293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:21.475201 17295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:21.475626 17285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:21.476306 17285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:21.477475 17285 hybrid_clock.cc:648] HybridClock initialized: now 1786213221477453 us; error 33 us; skew 500 ppm
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:21.480247 17285 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:20:21.481038 17285 webserver.cc:460] Webserver started at http://127.27.18.194:37321/ using document root <none> and password file <none>
I20260808 18:20:21.481549 17285 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:21.481662 17285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:21.481909 17285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:21.484028 17285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "71173bf6e0a24e18ae02154f77fa451b"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "5e19b3a9075c9ca772e74372a36546f7"
server_key_iv: "1344aef4877b18b6ff24d19fd3177ddc"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.484665 17285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "71173bf6e0a24e18ae02154f77fa451b"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "5e19b3a9075c9ca772e74372a36546f7"
server_key_iv: "1344aef4877b18b6ff24d19fd3177ddc"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.488242 17285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:20:21.490438 17312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.491554 17285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:21.491699 17285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "71173bf6e0a24e18ae02154f77fa451b"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "5e19b3a9075c9ca772e74372a36546f7"
server_key_iv: "1344aef4877b18b6ff24d19fd3177ddc"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.491806 17285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:21.513018 17285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:21.515836 17285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:21.516054 17285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:21.516747 17285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:21.517680 17285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:21.517776 17285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.517844 17285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:21.517902 17285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.527097 17285 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33373
I20260808 18:20:21.527118 17504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33373 every 8 connection(s)
I20260808 18:20:21.528002 17285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:20:21.535238 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 17285
I20260808 18:20:21.535431 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:20:21.535737 27723 external_mini_cluster.cc:1428] Setting key 743399832d76b68d58cd6958894f6cdd
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:21.541800 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.529710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:35775 (local address 127.27.18.254:38431)
0808 18:20:21.529842 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.529845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.530589 (+ 744us) server_negotiation.cc:407] Connection header received
0808 18:20:21.531909 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.531912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.531968 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.532073 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:21.533158 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.533676 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.534788 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.535006 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.535107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.537831 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:21.537837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:21.537842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:21.537875 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:21.539518 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.540168 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.540171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.540172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.540242 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.540668 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.540670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.540671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.540825 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:21.540898 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.541492 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.541622 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:20:21.542487 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.530040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:35775)
0808 18:20:21.530460 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.530496 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.531694 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.532248 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.532254 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.532576 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:21.533015 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.533027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.533813 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.533816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.534651 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.534659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.535159 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.535163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.535323 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.535956 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:21.535980 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:21.537649 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:21.539654 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:21.539663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:21.539679 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:21.540060 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:21.540337 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:21.540342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:21.540345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:21.540539 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:21.540933 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:21.540943 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:21.541243 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.541440 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.541749 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260808 18:20:21.543694 17505 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38431
I20260808 18:20:21.543931 17505 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:21.544533 17505 heartbeater.cc:511] Master 127.27.18.254:38431 requested a full tablet report, sending...
I20260808 18:20:21.545640 16980 ts_manager.cc:194] Registered new tserver with Master: 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
I20260808 18:20:21.546350 16980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:35775
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:20:21.564335 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38431
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:21.667853 17514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:21.668063 17514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:21.668130 17514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:21.672106 17514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:21.672153 17514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:21.672247 17514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:20:21.677175 17514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:21.678022 17514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:21.678879 17514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:21.688014 17520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:21.687999 17522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:21.688063 17514 server_base.cc:1034] running on GCE node
W20260808 18:20:21.688016 17525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:21.688660 17514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:21.689265 17514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:21.690423 17514 hybrid_clock.cc:648] HybridClock initialized: now 1786213221690411 us; error 34 us; skew 500 ppm
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:21.693074 17514 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:20:21.693836 17514 webserver.cc:460] Webserver started at http://127.27.18.195:43977/ using document root <none> and password file <none>
I20260808 18:20:21.694365 17514 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:21.694422 17514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:21.694656 17514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:21.696619 17514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "b29b46a67ee04915973f862ef2e597f9"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e13596a9c4a24d61829f88be1ee98f64"
server_key_iv: "108303bd3febd01d7f81415d8d077878"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.697153 17514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "b29b46a67ee04915973f862ef2e597f9"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e13596a9c4a24d61829f88be1ee98f64"
server_key_iv: "108303bd3febd01d7f81415d8d077878"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.700536 17514 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:20:21.702729 17537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.703893 17514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:21.704020 17514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "b29b46a67ee04915973f862ef2e597f9"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e13596a9c4a24d61829f88be1ee98f64"
server_key_iv: "108303bd3febd01d7f81415d8d077878"
server_key_version: "encryptionkey@0"
I20260808 18:20:21.704123 17514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:21.739188 17514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:21.742136 17514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:21.742322 17514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:21.742871 17514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:21.743664 17514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:21.743736 17514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.743806 17514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:21.743854 17514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:21.752894 17514 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46099
I20260808 18:20:21.752930 17717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46099 every 8 connection(s)
I20260808 18:20:21.753749 17514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:20:21.760432 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 17514
I20260808 18:20:21.760584 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:20:21.760872 27723 external_mini_cluster.cc:1428] Setting key cb1fbc83ee88674ba8b5a29434c3a54e
Aug 08 18:20:21 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213221, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:21.766373 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.755430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:46075 (local address 127.27.18.254:38431)
0808 18:20:21.755573 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.755576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.756329 (+ 753us) server_negotiation.cc:407] Connection header received
0808 18:20:21.757563 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.757566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.757612 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.757695 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:21.758700 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.759174 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.760205 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.760417 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.760490 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.763177 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:21.763182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:21.763185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:21.763215 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:21.764480 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.764970 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.764973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.764975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.765057 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.765402 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.765406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.765407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.765572 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:21.765669 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.766105 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.766216 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:20:21.767071 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.755764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:46075)
0808 18:20:21.756182 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.756221 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.757355 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.757850 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.757855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.758162 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:21.758584 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.758595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.759272 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.759274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.760072 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.760080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.760569 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.760573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.760761 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.761397 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:21.761413 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:21.762998 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:21.764603 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:21.764609 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:21.764622 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:21.764867 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:21.765151 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:21.765155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:21.765157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:21.765291 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:21.765683 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:21.765691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:21.765895 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.766062 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.766343 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
I20260808 18:20:21.768069 17721 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38431
I20260808 18:20:21.768285 17721 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:21.768711 17721 heartbeater.cc:511] Master 127.27.18.254:38431 requested a full tablet report, sending...
I20260808 18:20:21.769671 16980 ts_manager.cc:194] Registered new tserver with Master: b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099)
I20260808 18:20:21.770210 16980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:46075
I20260808 18:20:21.774622 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:20:21.782831 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.776440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.27.18.254:38431)
0808 18:20:21.776568 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.776572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.776722 (+ 150us) server_negotiation.cc:407] Connection header received
0808 18:20:21.776814 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.776816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.776848 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.776909 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:21.777526 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.777970 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.778722 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.778932 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.779033 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.780271 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:21.780274 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:21.780276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:21.780298 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:21.781315 (+ 1017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.781697 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.781701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.781702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.781777 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:21.782037 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:21.782040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:21.782042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:21.782242 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:21.782380 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.782637 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.782727 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:20:21.785742 16980 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:20:21.788623 16980 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:20:21.801673 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.796743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33373 (local address 127.0.0.1:49888)
0808 18:20:21.797749 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.797762 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.797853 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.798287 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.798290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.798318 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:21.798403 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.798409 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.799317 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.799322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.800442 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.800451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.801217 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.801316 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.801352 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.801451 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.801550 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260808 18:20:21.801718 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.797363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.27.18.194:33373)
0808 18:20:21.797713 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.797724 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.797850 (+ 126us) server_negotiation.cc:407] Connection header received
0808 18:20:21.797962 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.797968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.798137 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.798307 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:21.798500 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.799184 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.800559 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.801100 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801202 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.801331 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.801496 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.801581 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":120,"threads_started":1}
I20260808 18:20:21.802381 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.796114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:35539 (local address 127.0.0.1:50126)
0808 18:20:21.796742 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.796786 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.796917 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.798021 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.798023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.798047 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:21.798206 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.798212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.799640 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.799643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.800947 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.800958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.801977 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.802066 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.802087 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.802157 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.802228 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":150,"spinlock_wait_cycles":3200,"thread_start_us":66,"threads_started":1}
I20260808 18:20:21.802425 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.796273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.27.18.193:35539)
0808 18:20:21.797549 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.797566 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.797590 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:20:21.797679 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.797684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.797887 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.798037 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:21.798352 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.799498 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801086 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.801878 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801974 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.802092 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.802181 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.802270 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":115,"threads_started":1}
I20260808 18:20:21.802676 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.796920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.27.18.195:46099)
0808 18:20:21.797292 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.797304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.797327 (+ 23us) server_negotiation.cc:407] Connection header received
0808 18:20:21.797398 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.797404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.797599 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.797767 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:21.798338 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.799495 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801080 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.801812 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.801912 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.802029 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.802459 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.802558 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":192,"threads_started":1}
I20260808 18:20:21.802394 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.796317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46099 (local address 127.0.0.1:54334)
0808 18:20:21.797166 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.797179 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.797263 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.797966 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.797991 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.798016 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:21.798201 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.798208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.799640 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.799643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.800953 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.800958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.802078 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.802081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.802166 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.802182 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.802236 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.802283 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":180,"spinlock_wait_cycles":1408,"thread_start_us":94,"threads_started":1}
I20260808 18:20:21.805054 17396 tablet_service.cc:1467] Processing CreateTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba (DEFAULT_TABLE table=test-table [id=86b17a97bf684a7d8c0f0b975cd9c7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:21.805262 17165 tablet_service.cc:1467] Processing CreateTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba (DEFAULT_TABLE table=test-table [id=86b17a97bf684a7d8c0f0b975cd9c7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:21.806118 17165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0833d62f5b8a488c8c0ffb83382a80ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:21.806145 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0833d62f5b8a488c8c0ffb83382a80ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:21.806173 17605 tablet_service.cc:1467] Processing CreateTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba (DEFAULT_TABLE table=test-table [id=86b17a97bf684a7d8c0f0b975cd9c7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:21.807139 17605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0833d62f5b8a488c8c0ffb83382a80ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:21.812860 17755 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Bootstrap starting.
I20260808 18:20:21.813740 17757 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Bootstrap starting.
I20260808 18:20:21.814281 17756 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Bootstrap starting.
I20260808 18:20:21.815070 17755 tablet_bootstrap.cc:654] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:21.816324 17757 tablet_bootstrap.cc:654] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:21.816390 17755 log.cc:824] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:21.816952 17756 tablet_bootstrap.cc:654] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:21.817334 17757 log.cc:824] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:21.817757 17756 log.cc:824] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:21.818619 17755 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: No bootstrap required, opened a new log
I20260808 18:20:21.818821 17755 ts_tablet_manager.cc:1397] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:20:21.819234 17757 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: No bootstrap required, opened a new log
I20260808 18:20:21.819456 17757 ts_tablet_manager.cc:1397] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:20:21.819658 17756 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: No bootstrap required, opened a new log
I20260808 18:20:21.819835 17756 ts_tablet_manager.cc:1397] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:20:21.821913 17755 raft_consensus.cc:348] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.822410 17755 raft_consensus.cc:374] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:21.822502 17755 raft_consensus.cc:729] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59de6ece509c45e6ac435edc7178fc53, State: Initialized, Role: FOLLOWER
I20260808 18:20:21.823029 17755 consensus_queue.cc:260] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.823303 17757 raft_consensus.cc:348] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.823617 17757 raft_consensus.cc:374] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:21.823705 17757 raft_consensus.cc:729] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29b46a67ee04915973f862ef2e597f9, State: Initialized, Role: FOLLOWER
I20260808 18:20:21.823552 17756 raft_consensus.cc:348] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.823724 17274 heartbeater.cc:503] Master 127.27.18.254:38431 was elected leader, sending a full tablet report...
I20260808 18:20:21.823778 17756 raft_consensus.cc:374] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:21.823887 17756 raft_consensus.cc:729] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71173bf6e0a24e18ae02154f77fa451b, State: Initialized, Role: FOLLOWER
I20260808 18:20:21.824071 17755 ts_tablet_manager.cc:1428] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:20:21.824241 17757 consensus_queue.cc:260] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.824385 17756 consensus_queue.cc:260] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.825055 17505 heartbeater.cc:503] Master 127.27.18.254:38431 was elected leader, sending a full tablet report...
I20260808 18:20:21.825186 17757 ts_tablet_manager.cc:1428] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:20:21.825435 17756 ts_tablet_manager.cc:1428] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:20:21.825857 17721 heartbeater.cc:503] Master 127.27.18.254:38431 was elected leader, sending a full tablet report...
I20260808 18:20:21.910635 17770 raft_consensus.cc:482] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:21.910818 17770 raft_consensus.cc:504] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.911973 17770 leader_election.cc:288] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539), 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
I20260808 18:20:21.915555 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.912280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:57679 (local address 127.27.18.193:35539)
0808 18:20:21.912451 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.912456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.912477 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:20:21.912540 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.912544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.912605 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.912695 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:21.913000 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.913494 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.914575 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.914957 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.915088 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.915161 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.915355 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.915449 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:20:21.915555 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.912159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:35539 (local address 127.27.18.195:57679)
0808 18:20:21.912302 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.912325 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.912432 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.912720 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.912724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.912744 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:21.912871 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.912877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.913609 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.913613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.914444 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.914453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.915081 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.915085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.915208 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.915235 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.915326 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.915419 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":20}
I20260808 18:20:21.916164 17209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0833d62f5b8a488c8c0ffb83382a80ba" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59de6ece509c45e6ac435edc7178fc53" is_pre_election: true
I20260808 18:20:21.916247 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.912481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33373 (local address 127.27.18.195:44131)
0808 18:20:21.912930 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:21.912940 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:21.913056 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:21.913320 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:21.913323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:21.913340 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:21.913462 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.913467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.914184 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.914200 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.914989 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:21.914997 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.915855 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.915859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:21.915942 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.915959 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:20:21.916050 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:20:21.916112 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":17,"thread_start_us":151,"threads_started":1}
I20260808 18:20:21.916237 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.912597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:44131 (local address 127.27.18.194:33373)
0808 18:20:21.912752 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.912757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.913029 (+ 272us) server_negotiation.cc:407] Connection header received
0808 18:20:21.913181 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.913183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.913227 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.913317 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:21.913577 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.914054 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.915139 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.915635 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.915769 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.915822 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.916069 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.916138 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 18:20:21.916517 17209 raft_consensus.cc:2454] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 0.
I20260808 18:20:21.916908 17436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0833d62f5b8a488c8c0ffb83382a80ba" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71173bf6e0a24e18ae02154f77fa451b" is_pre_election: true
I20260808 18:20:21.917006 17543 leader_election.cc:302] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59de6ece509c45e6ac435edc7178fc53, b29b46a67ee04915973f862ef2e597f9; no voters:
I20260808 18:20:21.917218 17436 raft_consensus.cc:2454] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 0.
I20260808 18:20:21.917284 17770 raft_consensus.cc:2781] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:20:21.917407 17770 raft_consensus.cc:482] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:20:21.917459 17770 raft_consensus.cc:3037] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:21.918623 17770 raft_consensus.cc:504] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.918983 17770 leader_election.cc:288] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 election: Requested vote from peers 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539), 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
I20260808 18:20:21.919242 17209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0833d62f5b8a488c8c0ffb83382a80ba" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59de6ece509c45e6ac435edc7178fc53"
I20260808 18:20:21.919407 17436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0833d62f5b8a488c8c0ffb83382a80ba" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71173bf6e0a24e18ae02154f77fa451b"
I20260808 18:20:21.919421 17209 raft_consensus.cc:3037] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:21.919530 17436 raft_consensus.cc:3037] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:21.920735 17209 raft_consensus.cc:2454] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 1.
I20260808 18:20:21.920735 17436 raft_consensus.cc:2454] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 1.
I20260808 18:20:21.921003 17543 leader_election.cc:302] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9; no voters:
I20260808 18:20:21.921200 17770 raft_consensus.cc:2781] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:21.921402 17770 raft_consensus.cc:686] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 1 LEADER]: Becoming Leader. State: Replica: b29b46a67ee04915973f862ef2e597f9, State: Running, Role: LEADER
I20260808 18:20:21.921711 17770 consensus_queue.cc:237] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:21.924551 16980 catalog_manager.cc:5495] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b29b46a67ee04915973f862ef2e597f9 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "b29b46a67ee04915973f862ef2e597f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260808 18:20:21.956390 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:21.952760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.27.18.195:46099)
0808 18:20:21.952913 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:21.952918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:21.952932 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:20:21.953131 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:21.953134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:21.953189 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:21.953300 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:20:21.953554 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.954059 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.954913 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:21.955129 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:21.955238 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:21.955601 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:20:21.955995 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:20:21.956134 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:21.956199 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 18:20:21.956254 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 18:20:21.962016 17436 raft_consensus.cc:1261] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Refusing update from remote peer b29b46a67ee04915973f862ef2e597f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:21.962137 17209 raft_consensus.cc:1261] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Refusing update from remote peer b29b46a67ee04915973f862ef2e597f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:21.962808 17775 consensus_queue.cc:1035] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:21.963004 17770 consensus_queue.cc:1035] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:21.971390 17782 mvcc.cc:205] Tried to move back new op lower bound from 7316329357149020160 to 7316329356993724416. Current Snapshot: MvccSnapshot[applied={T|T < 7316329357149020160}]
I20260808 18:20:21.977696 16978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:21.977859 16978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:21.980201 16978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Sending DeleteTablet for 3 replicas of tablet 0833d62f5b8a488c8c0ffb83382a80ba
I20260808 18:20:21.980994 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 17075
I20260808 18:20:21.980989 17605 tablet_service.cc:1514] Processing DeleteTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:21 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:54334
I20260808 18:20:21.981088 17165 tablet_service.cc:1514] Processing DeleteTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:21 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50126
I20260808 18:20:21.981688 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:21 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:49888
I20260808 18:20:21.987798 17796 tablet_replica.cc:330] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: stopping tablet replica
I20260808 18:20:21.988245 17796 raft_consensus.cc:2229] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:21.988615 17796 raft_consensus.cc:2258] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:21.991598 17796 ts_tablet_manager.cc:1905] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:21.994935 17796 ts_tablet_manager.cc:1918] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:21.995065 17795 tablet_replica.cc:330] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: stopping tablet replica
I20260808 18:20:21.995201 17796 log.cc:1196] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/0833d62f5b8a488c8c0ffb83382a80ba
I20260808 18:20:21.995353 17795 raft_consensus.cc:2229] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:20:21.995711 17795 raft_consensus.cc:2258] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:21.995715 17796 ts_tablet_manager.cc:1939] T 0833d62f5b8a488c8c0ffb83382a80ba P 71173bf6e0a24e18ae02154f77fa451b: Deleting consensus metadata
I20260808 18:20:21.997193 17795 ts_tablet_manager.cc:1905] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:21.997174 16962 catalog_manager.cc:4841] TS 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373): tablet 0833d62f5b8a488c8c0ffb83382a80ba (table test-table [id=86b17a97bf684a7d8c0f0b975cd9c7e2]) successfully deleted
I20260808 18:20:21.999662 17795 ts_tablet_manager.cc:1918] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:21.999751 17795 log.cc:1196] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/0833d62f5b8a488c8c0ffb83382a80ba
I20260808 18:20:22.000087 17795 ts_tablet_manager.cc:1939] T 0833d62f5b8a488c8c0ffb83382a80ba P b29b46a67ee04915973f862ef2e597f9: Deleting consensus metadata
W20260808 18:20:22.000468 16962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:35539 (error 108)
I20260808 18:20:22.001044 16955 catalog_manager.cc:4841] TS b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099): tablet 0833d62f5b8a488c8c0ffb83382a80ba (table test-table [id=86b17a97bf684a7d8c0f0b975cd9c7e2]) successfully deleted
I20260808 18:20:22.001606 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.001322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:35539 (local address 127.0.0.1:50138)
0808 18:20:22.001453 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.001526 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:35539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 18:20:22.001705 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 17285
W20260808 18:20:22.001818 16962 catalog_manager.cc:4584] TS 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0833d62f5b8a488c8c0ffb83382a80ba: Network error: Client connection negotiation failed: client connection to 127.27.18.193:35539: connect: Connection refused (error 111)
I20260808 18:20:22.012058 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 17514
I20260808 18:20:22.021901 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 16931
I20260808 18:20:22.036415 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38431
--webserver_interface=127.27.18.254
--webserver_port=37009
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:38431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:22.139019 17798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:22.139221 17798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:22.139274 17798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:22.143054 17798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:22.143098 17798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:22.143115 17798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:22.143131 17798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:22.143144 17798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:22.147877 17798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:38431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=37009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:22.148613 17798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:22.149397 17798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:22.162073 17812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:22.162097 17807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.162258 17798 server_base.cc:1034] running on GCE node
W20260808 18:20:22.162442 17808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.162981 17798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:22.164162 17798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:22.165323 17798 hybrid_clock.cc:648] HybridClock initialized: now 1786213222165307 us; error 27 us; skew 500 ppm
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:22.168114 17798 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:22.168885 17798 webserver.cc:460] Webserver started at http://127.27.18.254:37009/ using document root <none> and password file <none>
I20260808 18:20:22.169395 17798 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:22.169463 17798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:22.172993 17798 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 18:20:22.174975 17822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.176222 17798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:20:22.176340 17798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5f8500f10896422995e47e9b17df89cc"
format_stamp: "Formatted at 2026-08-08 18:20:20 on dist-test-slave-x4qh"
server_key: "8ec1e9da1d3658f3171e81c82a1b2c72"
server_key_iv: "bdc384e1185efff034964b9e8bcc64f1"
server_key_version: "encryptionkey@0"
I20260808 18:20:22.176919 17798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:22.195072 17798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:22.198160 17798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:22.198292 17798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:22.205227 17798 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:38431
I20260808 18:20:22.205492 17902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:38431 every 8 connection(s)
I20260808 18:20:22.206244 17798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:22.210278 17903 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:20:22.211858 17903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap starting.
I20260808 18:20:22.212911 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 17798
I20260808 18:20:22.213352 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:35539
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:38431
--webserver_port=40737
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:20:22.233773 17903 log.cc:824] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:22.244169 17903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:20:22.244555 17903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap complete.
I20260808 18:20:22.249869 17903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:22.250357 17903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f8500f10896422995e47e9b17df89cc, State: Initialized, Role: FOLLOWER
I20260808 18:20:22.250834 17903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:22.250962 17903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:22.251031 17903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:22.251116 17903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:20:22.253314 17903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:22.253654 17903 leader_election.cc:302] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f8500f10896422995e47e9b17df89cc; no voters:
I20260808 18:20:22.254007 17903 leader_election.cc:288] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:20:22.254119 17908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:20:22.254390 17908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 LEADER]: Becoming Leader. State: Replica: 5f8500f10896422995e47e9b17df89cc, State: Running, Role: LEADER
I20260808 18:20:22.254664 17908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:22.255151 17903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:22.256273 17911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f8500f10896422995e47e9b17df89cc. Latest consensus state: current_term: 2 leader_uuid: "5f8500f10896422995e47e9b17df89cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } } }
I20260808 18:20:22.256429 17911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:22.256273 17910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5f8500f10896422995e47e9b17df89cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } } }
I20260808 18:20:22.256735 17910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:22.257136 17923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:22.261986 17923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:22.262545 17923 catalog_manager.cc:1259] Loaded cluster ID: d95c5f3493f6461b9736cc34ae49b00a
I20260808 18:20:22.262593 17923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:22.263970 17923 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 18:20:22.268491 17923 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 18:20:22.273684 17923 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:22.274317 17923 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Loaded TSK: 0
I20260808 18:20:22.275424 17923 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:20:22.337435 17906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:22.337656 17906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:22.337697 17906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:22.341509 17906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:22.341557 17906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:22.341607 17906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:20:22.346474 17906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:35539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=40737
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:22.347456 17906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:22.348479 17906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:22.357322 17949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:22.357347 17948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:22.357568 17952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.357728 17906 server_base.cc:1034] running on GCE node
I20260808 18:20:22.358129 17906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:22.358728 17906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:22.359911 17906 hybrid_clock.cc:648] HybridClock initialized: now 1786213222359883 us; error 45 us; skew 500 ppm
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:22.366708 17906 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:20:22.367714 17906 webserver.cc:460] Webserver started at http://127.27.18.193:40737/ using document root <none> and password file <none>
I20260808 18:20:22.368247 17906 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:22.368320 17906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:22.372153 17906 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:20:22.374372 17966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.375835 17906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:22.375967 17906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "59de6ece509c45e6ac435edc7178fc53"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e3e5d866a11fff2a0c8942e3d28f1924"
server_key_iv: "4ea87566d2d1330f52846673113f65b8"
server_key_version: "encryptionkey@0"
I20260808 18:20:22.376479 17906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:22.388475 17906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:22.391636 17906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:22.391819 17906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:22.392351 17906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:22.393358 17978 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:20:22.397418 17906 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:20:22.397488 17906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 18:20:22.397585 17906 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:20:22.399108 17906 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:20:22.399173 17906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 18:20:22.399470 17978 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Bootstrap starting.
I20260808 18:20:22.409379 17906 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:35539
I20260808 18:20:22.409416 18158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:35539 every 8 connection(s)
I20260808 18:20:22.410243 17906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:20:22.411607 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 17906
I20260808 18:20:22.412134 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:33373
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:38431
--webserver_port=37321
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:20:22.419545 17978 log.cc:824] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:22.426859 17978 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:20:22.427289 17978 tablet_bootstrap.cc:492] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Bootstrap complete.
I20260808 18:20:22.428628 17978 ts_tablet_manager.cc:1397] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.004s
I20260808 18:20:22.430330 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.412349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:40215)
0808 18:20:22.413393 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.413447 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.414356 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.415374 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.415381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.415913 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:22.416527 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.416565 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.418635 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.418642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.420258 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.420274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.421630 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.421635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.421861 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.422570 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:22.422593 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:22.424542 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:22.427094 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.427102 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.427116 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.427496 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.427919 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.427925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.427929 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.428092 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.428729 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:22.428735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:22.429003 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.429188 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.429509 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"thread_start_us":143,"threads_started":1}
I20260808 18:20:22.430351 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.412377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:40215 (local address 127.27.18.254:38431)
0808 18:20:22.413123 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.413149 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.413586 (+ 437us) server_negotiation.cc:407] Connection header received
0808 18:20:22.414744 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.414754 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.415007 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.415303 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:22.416748 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.418403 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.420431 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.421232 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.421379 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.424750 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:22.424756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:22.424769 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:22.424805 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:22.426949 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.427649 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.427655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.427661 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.427822 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.428196 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.428199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.428201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.428586 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:22.428728 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.429355 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.429631 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":102,"threads_started":1}
I20260808 18:20:22.431262 17978 raft_consensus.cc:348] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:22.431725 17978 raft_consensus.cc:729] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59de6ece509c45e6ac435edc7178fc53, State: Initialized, Role: FOLLOWER
I20260808 18:20:22.432242 17978 consensus_queue.cc:260] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } } peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } }
I20260808 18:20:22.433184 17978 ts_tablet_manager.cc:1428] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:20:22.433207 18159 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38431
I20260808 18:20:22.433436 18159 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:22.433945 18159 heartbeater.cc:511] Master 127.27.18.254:38431 requested a full tablet report, sending...
I20260808 18:20:22.436177 17852 ts_manager.cc:194] Registered new tserver with Master: 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539)
I20260808 18:20:22.439360 17852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:40215
I20260808 18:20:22.441540 18159 heartbeater.cc:503] Master 127.27.18.254:38431 was elected leader, sending a full tablet report...
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:20:22.448645 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.438787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.27.18.193:35539)
0808 18:20:22.439184 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.439194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.439780 (+ 586us) server_negotiation.cc:407] Connection header received
0808 18:20:22.439852 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.439857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.440025 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.440136 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:22.440893 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.441847 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.442905 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.443294 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.443431 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.445391 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:22.445400 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:22.445411 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:22.445432 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:22.446785 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.447257 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.447263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.447265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.447348 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.447650 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.447653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.447655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.447958 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:22.448077 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.448379 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.448520 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":320,"thread_start_us":107,"threads_started":1}
I20260808 18:20:22.448640 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.438707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:35539 (local address 127.0.0.1:50142)
0808 18:20:22.439456 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.439510 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.439650 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.440277 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.440283 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.440641 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:22.440751 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.440758 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.441958 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.441962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.442811 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.442822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.443422 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.443426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.443527 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.443982 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:22.443992 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:22.445250 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:22.446913 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.446921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.446926 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.447142 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.447470 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.447473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.447475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.447558 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.448055 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:22.448060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:22.448173 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.448310 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.448478 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":124,"threads_started":1}
I20260808 18:20:22.449849 18043 tablet_service.cc:1514] Processing DeleteTablet for tablet 0833d62f5b8a488c8c0ffb83382a80ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:21 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50142
I20260808 18:20:22.450686 18178 tablet_replica.cc:330] stopping tablet replica
I20260808 18:20:22.450959 18178 raft_consensus.cc:2229] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:22.451177 18178 raft_consensus.cc:2258] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:22.452049 18178 ts_tablet_manager.cc:1905] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:22.455245 18178 ts_tablet_manager.cc:1918] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:22.455320 18178 log.cc:1196] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/0833d62f5b8a488c8c0ffb83382a80ba
I20260808 18:20:22.455637 18178 ts_tablet_manager.cc:1939] T 0833d62f5b8a488c8c0ffb83382a80ba P 59de6ece509c45e6ac435edc7178fc53: Deleting consensus metadata
I20260808 18:20:22.457311 17827 catalog_manager.cc:4841] TS 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539): tablet 0833d62f5b8a488c8c0ffb83382a80ba (table test-table [id=86b17a97bf684a7d8c0f0b975cd9c7e2]) successfully deleted
W20260808 18:20:22.528098 18168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:22.528331 18168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:22.528409 18168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:22.532035 18168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:22.532102 18168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:22.532178 18168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:20:22.536752 18168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:33373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=37321
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:22.537637 18168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:22.538540 18168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:22.547550 18183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:22.547591 18184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.547641 18168 server_base.cc:1034] running on GCE node
W20260808 18:20:22.547591 18186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.548275 18168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:22.548786 18168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:22.549952 18168 hybrid_clock.cc:648] HybridClock initialized: now 1786213222549926 us; error 40 us; skew 500 ppm
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:22.552724 18168 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:20:22.553545 18168 webserver.cc:460] Webserver started at http://127.27.18.194:37321/ using document root <none> and password file <none>
I20260808 18:20:22.554065 18168 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:22.554132 18168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:22.558040 18168 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:22.559890 18198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.560978 18168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:22.561112 18168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "71173bf6e0a24e18ae02154f77fa451b"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "5e19b3a9075c9ca772e74372a36546f7"
server_key_iv: "1344aef4877b18b6ff24d19fd3177ddc"
server_key_version: "encryptionkey@0"
I20260808 18:20:22.561619 18168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:22.572341 18168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:22.575163 18168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:22.575343 18168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:22.576009 18168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:22.576903 18168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:22.576951 18168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.577016 18168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:22.577052 18168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.585809 18168 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33373
I20260808 18:20:22.585835 18376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33373 every 8 connection(s)
I20260808 18:20:22.586717 18168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:20:22.589983 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 18168
I20260808 18:20:22.590385 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:46099
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:38431
--webserver_port=43977
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:22.600836 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.588421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:36911 (local address 127.27.18.254:38431)
0808 18:20:22.588583 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.588589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.589544 (+ 955us) server_negotiation.cc:407] Connection header received
0808 18:20:22.590988 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.590995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.591067 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.591165 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:22.592562 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.593116 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.594499 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.594715 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.594800 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.597554 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:22.597557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:22.597559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:22.597581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:22.598917 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.599411 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.599415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.599418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.599492 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.599837 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.599839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.599841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.600018 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:22.600120 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.600552 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.600676 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:20:22.601505 18379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.588826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:36911)
0808 18:20:22.589401 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.589440 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.590776 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.591335 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.591344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.591793 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:22.592353 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.592373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.593254 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.593256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.594380 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.594394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.594873 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.594879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.595113 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.595719 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:22.595743 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:22.597383 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:22.599030 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.599035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.599047 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.599281 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.599586 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.599590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.599591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.599718 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.600121 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:22.600128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:22.600345 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.600512 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.600785 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260808 18:20:22.602559 18377 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38431
I20260808 18:20:22.602788 18377 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:22.603223 18377 heartbeater.cc:511] Master 127.27.18.254:38431 requested a full tablet report, sending...
I20260808 18:20:22.604285 17852 ts_manager.cc:194] Registered new tserver with Master: 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
I20260808 18:20:22.605039 17852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:36911
W20260808 18:20:22.698215 18381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:22.698427 18381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:22.698465 18381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:22.702204 18381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:22.702246 18381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:22.702297 18381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:20:22.706998 18381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:46099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=43977
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:22.707916 18381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:22.708778 18381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:22.717525 18393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:22.717546 18395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.717739 18381 server_base.cc:1034] running on GCE node
W20260808 18:20:22.717775 18389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:22.718274 18381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:22.718849 18381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:22.720054 18381 hybrid_clock.cc:648] HybridClock initialized: now 1786213222720029 us; error 37 us; skew 500 ppm
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:22.722548 18381 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:20:22.723259 18381 webserver.cc:460] Webserver started at http://127.27.18.195:43977/ using document root <none> and password file <none>
I20260808 18:20:22.723791 18381 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:22.723867 18381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:22.727154 18381 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:22.729029 18406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.730085 18381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:20:22.730197 18381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "b29b46a67ee04915973f862ef2e597f9"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e13596a9c4a24d61829f88be1ee98f64"
server_key_iv: "108303bd3febd01d7f81415d8d077878"
server_key_version: "encryptionkey@0"
I20260808 18:20:22.730654 18381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:22.745200 18381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:22.747874 18381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:22.748062 18381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:22.748670 18381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:22.749454 18381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:22.749498 18381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.749568 18381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:22.749620 18381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:22.759244 18381 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46099
I20260808 18:20:22.759326 18587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46099 every 8 connection(s)
I20260808 18:20:22.760207 18381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:20:22.768198 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 18381
Aug 08 18:20:22 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213222, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:22.773725 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.761881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:38357 (local address 127.27.18.254:38431)
0808 18:20:22.762032 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.762037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.762847 (+ 810us) server_negotiation.cc:407] Connection header received
0808 18:20:22.764171 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.764174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.764235 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.764310 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:22.765509 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.765998 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.767031 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.767228 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.767309 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.769917 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:22.769921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:22.769923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:22.769948 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:22.771644 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.772146 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.772149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.772150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.772222 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.772580 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.772583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.772586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.772819 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:22.772903 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.773425 (+ 522us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.773550 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:20:22.774402 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.762263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:38357)
0808 18:20:22.762698 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.762737 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.763992 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.764483 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.764489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.764835 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:22.765371 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.765397 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.766100 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.766102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.766930 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.766939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.767350 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.767353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.767529 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.768052 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:22.768067 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:22.769760 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:22.771752 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.771757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.771767 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.772023 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.772304 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:22.772307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:22.772309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:22.772488 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:22.772934 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:22.772941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:22.773183 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.773332 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.773548 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
I20260808 18:20:22.775605 18589 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38431
I20260808 18:20:22.775857 18589 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:22.776355 18589 heartbeater.cc:511] Master 127.27.18.254:38431 requested a full tablet report, sending...
I20260808 18:20:22.777369 17852 ts_manager.cc:194] Registered new tserver with Master: b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099)
I20260808 18:20:22.777842 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.768889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.27.18.254:38431)
0808 18:20:22.769131 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.769136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.769315 (+ 179us) server_negotiation.cc:407] Connection header received
0808 18:20:22.769469 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.769472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.769517 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.769608 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:22.770288 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.770891 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.772100 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.772395 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.772498 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.773947 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:22.773950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:22.773951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:22.773970 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:22.775708 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.776181 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.776185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.776186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.776265 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:22.776556 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:22.776560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:22.776562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:22.776750 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:22.776970 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.777102 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.777720 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":76,"threads_started":1}
I20260808 18:20:22.778079 17852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:38357
I20260808 18:20:22.779623 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:20:22.783308 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.780267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.27.18.254:38431)
0808 18:20:22.780370 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.780374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.780538 (+ 164us) server_negotiation.cc:407] Connection header received
0808 18:20:22.780673 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.780675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.780747 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.780824 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:20:22.781024 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.781487 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.782212 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.782415 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.782485 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.782712 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:20:22.783028 (+ 316us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:20:22.783115 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.783167 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.783204 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 18:20:22.783910 17852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:20:22.786753 17852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:20:22.794078 18043 tablet_service.cc:1467] Processing CreateTablet for tablet 7abadb447305465597bda98b49b647ad (DEFAULT_TABLE table=test-table [id=80bf75b9dc6a40f9a324aa4163f7eb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:22.794710 18043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abadb447305465597bda98b49b647ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:22.797600 17978 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Bootstrap starting.
I20260808 18:20:22.797801 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.792929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46099 (local address 127.0.0.1:54350)
0808 18:20:22.793057 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.793073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.793139 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.794344 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.794347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.794359 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:22.794420 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.794424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.795450 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.795457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.796548 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.796558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.797352 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.797356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.797443 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.797479 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.797593 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.797671 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":28}
I20260808 18:20:22.797857 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.793327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54350 (local address 127.27.18.195:46099)
0808 18:20:22.793988 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.793996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.794012 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:20:22.794062 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.794065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.794185 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.794555 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:22.794661 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.795301 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.796699 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.797229 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.797351 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.797483 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.797625 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.797710 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":137,"threads_started":1}
I20260808 18:20:22.798461 17978 tablet_bootstrap.cc:654] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:22.799998 17978 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: No bootstrap required, opened a new log
I20260808 18:20:22.800084 17978 ts_tablet_manager.cc:1397] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 18:20:22.800345 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.793179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33373 (local address 127.0.0.1:49892)
0808 18:20:22.793374 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.793389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.793472 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.794279 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.794285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.794317 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:22.794406 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.794413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.795669 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.795672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.796942 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.796953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.799962 (+ 3009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.799966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.800052 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.800074 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.800159 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.800222 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260808 18:20:22.800421 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.793448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.27.18.194:33373)
0808 18:20:22.793816 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.793828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.793852 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:20:22.793926 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.793931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.794112 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.794281 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:22.794503 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.795513 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.797098 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.799833 (+ 2735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.799949 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.800084 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.800193 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.800288 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":133,"threads_started":1}
I20260808 18:20:22.800531 17978 raft_consensus.cc:348] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.800644 17978 raft_consensus.cc:374] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:22.800681 17978 raft_consensus.cc:729] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59de6ece509c45e6ac435edc7178fc53, State: Initialized, Role: FOLLOWER
I20260808 18:20:22.800805 17978 consensus_queue.cc:260] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.801082 17978 ts_tablet_manager.cc:1428] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 18:20:22.800928 18491 tablet_service.cc:1467] Processing CreateTablet for tablet 7abadb447305465597bda98b49b647ad (DEFAULT_TABLE table=test-table [id=80bf75b9dc6a40f9a324aa4163f7eb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:22.802104 18491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abadb447305465597bda98b49b647ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:22.803336 18275 tablet_service.cc:1467] Processing CreateTablet for tablet 7abadb447305465597bda98b49b647ad (DEFAULT_TABLE table=test-table [id=80bf75b9dc6a40f9a324aa4163f7eb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:22.804311 18275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7abadb447305465597bda98b49b647ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:22.809214 18604 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Bootstrap starting.
I20260808 18:20:22.810534 18605 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Bootstrap starting.
I20260808 18:20:22.811836 18604 tablet_bootstrap.cc:654] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:22.812778 18605 tablet_bootstrap.cc:654] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:22.812770 18604 log.cc:824] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:22.813354 18605 log.cc:824] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:22.814495 18604 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: No bootstrap required, opened a new log
I20260808 18:20:22.814699 18604 ts_tablet_manager.cc:1397] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:20:22.814977 18605 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: No bootstrap required, opened a new log
I20260808 18:20:22.815140 18605 ts_tablet_manager.cc:1397] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:20:22.817345 18604 raft_consensus.cc:348] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.817559 18604 raft_consensus.cc:374] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:22.817647 18604 raft_consensus.cc:729] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29b46a67ee04915973f862ef2e597f9, State: Initialized, Role: FOLLOWER
I20260808 18:20:22.817727 18605 raft_consensus.cc:348] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.817967 18605 raft_consensus.cc:374] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:22.818023 18605 raft_consensus.cc:729] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71173bf6e0a24e18ae02154f77fa451b, State: Initialized, Role: FOLLOWER
I20260808 18:20:22.818084 18604 consensus_queue.cc:260] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.818394 18605 consensus_queue.cc:260] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.818753 18589 heartbeater.cc:503] Master 127.27.18.254:38431 was elected leader, sending a full tablet report...
I20260808 18:20:22.818993 18377 heartbeater.cc:503] Master 127.27.18.254:38431 was elected leader, sending a full tablet report...
I20260808 18:20:22.819083 18604 ts_tablet_manager.cc:1428] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:20:22.819314 18605 ts_tablet_manager.cc:1428] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:20:22.829259 18609 raft_consensus.cc:482] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:22.829388 18609 raft_consensus.cc:504] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.830289 18609 leader_election.cc:288] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373), 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539)
I20260808 18:20:22.834349 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.830612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:53235 (local address 127.27.18.194:33373)
0808 18:20:22.830735 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.830739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.830752 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:20:22.830804 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.830806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.830847 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.831224 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:22.831325 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.831925 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.833314 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.833860 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.833941 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.833975 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.834190 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.834239 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:20:22.834369 18591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.830503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33373 (local address 127.27.18.195:53235)
0808 18:20:22.830616 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.830630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.830707 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.830978 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.830981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.830998 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:22.831120 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.831127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.832024 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.832028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.833163 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.833173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.833961 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.833965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.834082 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.834106 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.834179 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.834245 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 18:20:22.834807 18615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.830751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:35539 (local address 127.27.18.195:35941)
0808 18:20:22.831047 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.831059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:22.831161 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:22.831445 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:22.831448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:22.831464 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:22.831567 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.831572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.832395 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.832397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.833608 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:22.833618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.834373 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.834377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:22.834500 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.834520 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 18:20:22.834618 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:20:22.834680 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
I20260808 18:20:22.834842 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.830849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:35941 (local address 127.27.18.193:35539)
0808 18:20:22.830999 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.831003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.831131 (+ 128us) server_negotiation.cc:407] Connection header received
0808 18:20:22.831256 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.831258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.831301 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.831437 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:22.831668 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.832254 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.833730 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.834247 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.834350 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.834486 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.834638 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.834735 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 18:20:22.835078 18302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abadb447305465597bda98b49b647ad" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71173bf6e0a24e18ae02154f77fa451b" is_pre_election: true
I20260808 18:20:22.835320 18302 raft_consensus.cc:2454] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 0.
I20260808 18:20:22.835754 18420 leader_election.cc:302] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9; no voters:
I20260808 18:20:22.835844 18084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abadb447305465597bda98b49b647ad" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59de6ece509c45e6ac435edc7178fc53" is_pre_election: true
I20260808 18:20:22.835994 18609 raft_consensus.cc:2781] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:20:22.836071 18609 raft_consensus.cc:482] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:20:22.836118 18609 raft_consensus.cc:3037] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:22.836118 18084 raft_consensus.cc:2454] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 0.
I20260808 18:20:22.837143 18609 raft_consensus.cc:504] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.837476 18609 leader_election.cc:288] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 election: Requested vote from peers 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373), 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539)
W20260808 18:20:22.837637 18378 tablet.cc:2369] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:20:22.837812 18302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abadb447305465597bda98b49b647ad" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71173bf6e0a24e18ae02154f77fa451b"
I20260808 18:20:22.837906 18302 raft_consensus.cc:3037] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:22.837882 18084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7abadb447305465597bda98b49b647ad" candidate_uuid: "b29b46a67ee04915973f862ef2e597f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59de6ece509c45e6ac435edc7178fc53"
I20260808 18:20:22.837987 18084 raft_consensus.cc:3037] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:22.838750 18302 raft_consensus.cc:2454] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 1.
I20260808 18:20:22.838968 18084 raft_consensus.cc:2454] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29b46a67ee04915973f862ef2e597f9 in term 1.
I20260808 18:20:22.839030 18420 leader_election.cc:302] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9; no voters:
I20260808 18:20:22.839246 18609 raft_consensus.cc:2781] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:22.839497 18609 raft_consensus.cc:686] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 1 LEADER]: Becoming Leader. State: Replica: b29b46a67ee04915973f862ef2e597f9, State: Running, Role: LEADER
I20260808 18:20:22.839772 18609 consensus_queue.cc:237] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:22.842353 17848 catalog_manager.cc:5495] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b29b46a67ee04915973f862ef2e597f9 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "b29b46a67ee04915973f862ef2e597f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } health_report { overall_health: UNKNOWN } } }
I20260808 18:20:22.862958 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.859768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.27.18.195:46099)
0808 18:20:22.859897 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:22.859900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:22.859912 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:20:22.860036 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:22.860038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:22.860082 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:22.860186 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:20:22.860553 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.860993 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.861790 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:22.861970 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:22.862038 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:22.862290 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:20:22.862634 (+ 344us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:20:22.862765 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:22.862794 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 18:20:22.862838 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 18:20:22.868315 18302 raft_consensus.cc:1261] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Refusing update from remote peer b29b46a67ee04915973f862ef2e597f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:22.868268 18084 raft_consensus.cc:1261] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Refusing update from remote peer b29b46a67ee04915973f862ef2e597f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:22.868948 18609 consensus_queue.cc:1035] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:22.869112 18616 consensus_queue.cc:1035] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:22.874763 18621 mvcc.cc:205] Tried to move back new op lower bound from 7316329360861999104 to 7316329360754188288. Current Snapshot: MvccSnapshot[applied={T|T < 7316329360861999104}]
I20260808 18:20:22.875249 18622 mvcc.cc:205] Tried to move back new op lower bound from 7316329360861999104 to 7316329360754188288. Current Snapshot: MvccSnapshot[applied={T|T < 7316329360861999104}]
I20260808 18:20:22.876188 18625 mvcc.cc:205] Tried to move back new op lower bound from 7316329360861999104 to 7316329360754188288. Current Snapshot: MvccSnapshot[applied={T|T < 7316329360861999104}]
I20260808 18:20:22.883721 17852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:22.883889 17852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:22.886137 17852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Sending DeleteTablet for 3 replicas of tablet 7abadb447305465597bda98b49b647ad
I20260808 18:20:22.886770 18491 tablet_service.cc:1514] Processing DeleteTablet for tablet 7abadb447305465597bda98b49b647ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:22 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:54350
I20260808 18:20:22.886833 18043 tablet_service.cc:1514] Processing DeleteTablet for tablet 7abadb447305465597bda98b49b647ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:22 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50142
I20260808 18:20:22.886900 18275 tablet_service.cc:1514] Processing DeleteTablet for tablet 7abadb447305465597bda98b49b647ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:22 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:49892
I20260808 18:20:22.887027 18178 tablet_replica.cc:330] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: stopping tablet replica
I20260808 18:20:22.887163 18178 raft_consensus.cc:2229] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:22.887251 18644 tablet_replica.cc:330] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: stopping tablet replica
I20260808 18:20:22.887284 18178 raft_consensus.cc:2258] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:22.887565 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 17906
I20260808 18:20:22.887573 18644 raft_consensus.cc:2229] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:20:22.887919 18644 raft_consensus.cc:2258] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:22.890205 18645 tablet_replica.cc:330] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: stopping tablet replica
I20260808 18:20:22.890413 18645 raft_consensus.cc:2229] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:22.890611 18645 raft_consensus.cc:2258] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:22.893466 18645 ts_tablet_manager.cc:1905] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:22.896837 18645 ts_tablet_manager.cc:1918] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:22.896935 18645 log.cc:1196] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/7abadb447305465597bda98b49b647ad
I20260808 18:20:22.897262 18645 ts_tablet_manager.cc:1939] T 7abadb447305465597bda98b49b647ad P 71173bf6e0a24e18ae02154f77fa451b: Deleting consensus metadata
I20260808 18:20:22.898440 17827 catalog_manager.cc:4841] TS 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373): tablet 7abadb447305465597bda98b49b647ad (table test-table [id=80bf75b9dc6a40f9a324aa4163f7eb97]) successfully deleted
I20260808 18:20:22.903852 18644 ts_tablet_manager.cc:1905] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:22.905970 18644 ts_tablet_manager.cc:1918] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:22.906080 18644 log.cc:1196] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/7abadb447305465597bda98b49b647ad
I20260808 18:20:22.906473 18644 ts_tablet_manager.cc:1939] T 7abadb447305465597bda98b49b647ad P b29b46a67ee04915973f862ef2e597f9: Deleting consensus metadata
I20260808 18:20:22.907524 17823 catalog_manager.cc:4841] TS b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099): tablet 7abadb447305465597bda98b49b647ad (table test-table [id=80bf75b9dc6a40f9a324aa4163f7eb97]) successfully deleted
W20260808 18:20:22.910384 17827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:35539 (error 108)
I20260808 18:20:22.910933 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 18168
I20260808 18:20:22.911552 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:22.911133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:35539 (local address 127.0.0.1:50148)
0808 18:20:22.911317 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:22.911449 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:35539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 18:20:22.912717 17827 catalog_manager.cc:4584] TS 59de6ece509c45e6ac435edc7178fc53 (127.27.18.193:35539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7abadb447305465597bda98b49b647ad: Network error: Client connection negotiation failed: client connection to 127.27.18.193:35539: connect: Connection refused (error 111)
I20260808 18:20:22.923908 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 18381
I20260808 18:20:22.933539 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 17798
I20260808 18:20:22.944509 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38431
--webserver_interface=127.27.18.254
--webserver_port=37009
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:38431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:23.051198 18648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:23.051474 18648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:23.051519 18648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:23.055169 18648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:23.055212 18648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:23.055228 18648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:23.055260 18648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:23.055275 18648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:23.060098 18648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:38431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=37009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:23.060971 18648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:23.061906 18648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:23.069952 18653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:23.070005 18657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.070324 18648 server_base.cc:1034] running on GCE node
W20260808 18:20:23.069950 18654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.070926 18648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:23.071976 18648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:23.073149 18648 hybrid_clock.cc:648] HybridClock initialized: now 1786213223073133 us; error 27 us; skew 500 ppm
Aug 08 18:20:23 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213223, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:23.075650 18648 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:23.076529 18648 webserver.cc:460] Webserver started at http://127.27.18.254:37009/ using document root <none> and password file <none>
I20260808 18:20:23.077069 18648 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:23.077160 18648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:23.080621 18648 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:23.082473 18666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.083562 18648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:23.083688 18648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "5f8500f10896422995e47e9b17df89cc"
format_stamp: "Formatted at 2026-08-08 18:20:20 on dist-test-slave-x4qh"
server_key: "8ec1e9da1d3658f3171e81c82a1b2c72"
server_key_iv: "bdc384e1185efff034964b9e8bcc64f1"
server_key_version: "encryptionkey@0"
I20260808 18:20:23.084198 18648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:23.105495 18648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:23.108124 18648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:23.108315 18648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:23.115062 18648 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:38431
I20260808 18:20:23.115283 18757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:38431 every 8 connection(s)
I20260808 18:20:23.116117 18648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:23.120220 18759 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:20:23.121083 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 18648
I20260808 18:20:23.121507 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:35539
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:38431
--webserver_port=40737
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:20:23.121685 18759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap starting.
I20260808 18:20:23.144500 18759 log.cc:824] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:23.156026 18759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:20:23.156381 18759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc: Bootstrap complete.
I20260808 18:20:23.160331 18759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:23.160658 18759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f8500f10896422995e47e9b17df89cc, State: Initialized, Role: FOLLOWER
I20260808 18:20:23.161098 18759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:23.161206 18759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:23.161301 18759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:23.161399 18759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:20:23.163159 18759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:23.163502 18759 leader_election.cc:302] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f8500f10896422995e47e9b17df89cc; no voters:
I20260808 18:20:23.163813 18759 leader_election.cc:288] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 18:20:23.163882 18763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:20:23.164075 18763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [term 3 LEADER]: Becoming Leader. State: Replica: 5f8500f10896422995e47e9b17df89cc, State: Running, Role: LEADER
I20260808 18:20:23.164413 18763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } }
I20260808 18:20:23.164970 18759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:23.165307 18763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f8500f10896422995e47e9b17df89cc. Latest consensus state: current_term: 3 leader_uuid: "5f8500f10896422995e47e9b17df89cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } } }
I20260808 18:20:23.165419 18763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:23.165787 18764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5f8500f10896422995e47e9b17df89cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8500f10896422995e47e9b17df89cc" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38431 } } }
I20260808 18:20:23.165954 18764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8500f10896422995e47e9b17df89cc [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:23.166244 18769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:23.170471 18769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:23.170966 18769 catalog_manager.cc:1259] Loaded cluster ID: d95c5f3493f6461b9736cc34ae49b00a
I20260808 18:20:23.171041 18769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 18:20:23.171972 18769 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f13f43ed9ad google::LogMessage::Fail() at ??:0
@ 0x7f13f43f088a google::LogMessage::SendToLog() at ??:0
@ 0x7f13f43ed4de google::LogMessage::Flush() at ??:0
@ 0x7f13f43eea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f13fed2cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f13fed2dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f13fed2a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f13fed6e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f13f663f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f13f663fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f13f66413f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f13f6630512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f13f7c746db start_thread at ??:0
@ 0x7f13f2eeb61f clone at ??:0
W20260808 18:20:23.253746 18761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:23.254019 18761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:23.254119 18761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:23.258074 18761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:23.258170 18761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:23.258275 18761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:20:23.263252 18761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:35539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=40737
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:23.264263 18761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:23.265230 18761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:23.274374 18820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:23.274358 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.274700 18761 server_base.cc:1034] running on GCE node
W20260808 18:20:23.274770 18826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.275178 18761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:23.275799 18761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:23.276976 18761 hybrid_clock.cc:648] HybridClock initialized: now 1786213223276910 us; error 83 us; skew 500 ppm
Aug 08 18:20:23 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213223, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:23.279897 18761 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:20:23.280718 18761 webserver.cc:460] Webserver started at http://127.27.18.193:40737/ using document root <none> and password file <none>
I20260808 18:20:23.281244 18761 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:23.281306 18761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:23.285856 18761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:20:23.288079 18834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.289317 18761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:23.289446 18761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "59de6ece509c45e6ac435edc7178fc53"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e3e5d866a11fff2a0c8942e3d28f1924"
server_key_iv: "4ea87566d2d1330f52846673113f65b8"
server_key_version: "encryptionkey@0"
I20260808 18:20:23.289934 18761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:23.311528 18761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:23.314591 18761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:23.314821 18761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:23.315474 18761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:23.316624 18847 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:20:23.320773 18761 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:20:23.320832 18761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260808 18:20:23.320873 18761 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:20:23.322460 18761 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:20:23.322520 18761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 18:20:23.322804 18847 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Bootstrap starting.
I20260808 18:20:23.333397 18761 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:35539
I20260808 18:20:23.333757 19013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:35539 every 8 connection(s)
I20260808 18:20:23.334611 18761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:20:23.339665 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 18761
I20260808 18:20:23.340164 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:33373
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:38431
--webserver_port=37321
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:20:23.344216 18847 log.cc:824] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:23.353767 18847 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:20:23.354187 18847 tablet_bootstrap.cc:492] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Bootstrap complete.
I20260808 18:20:23.355707 18847 ts_tablet_manager.cc:1397] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.011s
I20260808 18:20:23.358647 18847 raft_consensus.cc:348] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:23.359117 18847 raft_consensus.cc:729] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59de6ece509c45e6ac435edc7178fc53, State: Initialized, Role: FOLLOWER
I20260808 18:20:23.359596 18847 consensus_queue.cc:260] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:23.360512 18847 ts_tablet_manager.cc:1428] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260808 18:20:23.393096 19016 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:20:23.337175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:52953)
0808 18:20:23.337872 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.337924 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:23.339103 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:23.392400 (+ 53297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":233,"threads_started":1}
W20260808 18:20:23.393697 19014 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:38431 (0 consecutive failures): Network error: Failed to ping master at 127.27.18.254:38431: Client connection negotiation failed: client connection to 127.27.18.254:38431: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:20:23.394513 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.394189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:47987)
0808 18:20:23.394374 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.394450 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 18:20:23.395201 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.394976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:54925)
0808 18:20:23.395093 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.395145 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:20:23.395447 19014 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:20:23.453923 19017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:23.454136 19017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:23.454173 19017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:23.457834 19017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:23.457906 19017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:23.457993 19017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:20:23.462548 19017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:33373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=37321
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:23.463425 19017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:23.464372 19017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:23.473608 19029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:23.473629 19026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:23.473642 19032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.473852 19017 server_base.cc:1034] running on GCE node
I20260808 18:20:23.474272 19017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:23.474884 19017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:23.476064 19017 hybrid_clock.cc:648] HybridClock initialized: now 1786213223476042 us; error 40 us; skew 500 ppm
Aug 08 18:20:23 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213223, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:23.478608 19017 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:20:23.479281 19017 webserver.cc:460] Webserver started at http://127.27.18.194:37321/ using document root <none> and password file <none>
I20260808 18:20:23.479795 19017 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:23.479853 19017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:23.483067 19017 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:20:23.484929 19044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.486016 19017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:20:23.486127 19017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "71173bf6e0a24e18ae02154f77fa451b"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "5e19b3a9075c9ca772e74372a36546f7"
server_key_iv: "1344aef4877b18b6ff24d19fd3177ddc"
server_key_version: "encryptionkey@0"
I20260808 18:20:23.486578 19017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:23.502947 19017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:23.505463 19017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:23.505667 19017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:23.506407 19017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:23.507531 19017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:23.507601 19017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.507687 19017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:23.507730 19017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.518520 19017 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33373
I20260808 18:20:23.518544 19204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33373 every 8 connection(s)
I20260808 18:20:23.519500 19017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:20:23.523423 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.521438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:44341)
0808 18:20:23.522025 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.522528 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":155,"threads_started":1}
W20260808 18:20:23.523841 19208 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:38431 (0 consecutive failures): Network error: Failed to ping master at 127.27.18.254:38431: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
I20260808 18:20:23.524384 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.524160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:33631)
0808 18:20:23.524270 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.524318 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 18:20:23.524948 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.524744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:48441)
0808 18:20:23.524854 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.524896 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 18:20:23.525106 19208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:20:23.528991 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 19017
I20260808 18:20:23.529376 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:46099
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:38431
--webserver_port=43977
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:23.633317 19212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:23.633550 19212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:23.633620 19212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:23.637181 19212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:23.637259 19212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:23.637347 19212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:20:23.641983 19212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:46099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=43977
--tserver_master_addrs=127.27.18.254:38431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:23.642822 19212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:23.643685 19212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:23.652765 19228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:23.652849 19226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.652982 19212 server_base.cc:1034] running on GCE node
W20260808 18:20:23.652781 19225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:23.653438 19212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:23.654009 19212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:23.655195 19212 hybrid_clock.cc:648] HybridClock initialized: now 1786213223655176 us; error 29 us; skew 500 ppm
Aug 08 18:20:23 dist-test-slave-x4qh krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213223, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:23.657802 19212 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:20:23.658581 19212 webserver.cc:460] Webserver started at http://127.27.18.195:43977/ using document root <none> and password file <none>
I20260808 18:20:23.659087 19212 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:23.659188 19212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:23.662467 19212 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:23.664327 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.665401 19212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:20:23.665542 19212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "b29b46a67ee04915973f862ef2e597f9"
format_stamp: "Formatted at 2026-08-08 18:20:21 on dist-test-slave-x4qh"
server_key: "e13596a9c4a24d61829f88be1ee98f64"
server_key_iv: "108303bd3febd01d7f81415d8d077878"
server_key_version: "encryptionkey@0"
I20260808 18:20:23.666055 19212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:23.679284 19212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:23.681700 19212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:23.681877 19212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:23.682379 19212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:23.683082 19212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:23.683149 19212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.683212 19212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:23.683266 19212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:23.691865 19212 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:46099
I20260808 18:20:23.691905 19416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:46099 every 8 connection(s)
I20260808 18:20:23.692693 19212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:20:23.696249 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 19212
I20260808 18:20:23.696408 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.694550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:47163)
0808 18:20:23.695017 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.695540 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":108,"threads_started":1}
W20260808 18:20:23.696898 19418 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:38431 (0 consecutive failures): Network error: Failed to ping master at 127.27.18.254:38431: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
I20260808 18:20:23.697535 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.697282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:41995)
0808 18:20:23.697417 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.697472 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 18:20:23.698094 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:23.697920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:40681)
0808 18:20:23.698009 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:23.698047 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 18:20:23.698242 19418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:20:24.396344 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:24.395966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:41847)
0808 18:20:24.396192 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:24.396243 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260808 18:20:24.526190 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:24.525690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:40203)
0808 18:20:24.525987 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:24.526056 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260808 18:20:24.699156 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:24.698800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:46925)
0808 18:20:24.699017 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:24.699066 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":66,"threads_started":1}
I20260808 18:20:25.202486 19430 raft_consensus.cc:482] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:25.202731 19430 raft_consensus.cc:504] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:25.203997 19430 leader_election.cc:288] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099), 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
Aug 08 18:20:25 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213223, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.195@KRBTEST.COM
Aug 08 18:20:25 dist-test-slave-x4qh krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213223, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.194@KRBTEST.COM
I20260808 18:20:25.220018 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.204193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33373 (local address 127.27.18.193:58705)
0808 18:20:25.204556 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:25.204570 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:25.204674 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:25.206604 (+ 1930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:25.206610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:25.207272 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:25.208146 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:25.208167 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.209988 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.209991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:25.211666 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:25.211677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.212636 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.212640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:25.212997 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:25.213726 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:25.213741 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:25.215809 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:25.217675 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:25.217679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:25.217682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:25.217963 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:25.218318 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:25.218321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:25.218323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:25.218465 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:25.219063 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:25.219081 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:25.219318 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 18:20:25.219526 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 18:20:25.219844 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":535,"spinlock_wait_cycles":44160,"thread_start_us":48,"threads_started":1}
I20260808 18:20:25.220036 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.204415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:58705 (local address 127.27.18.194:33373)
0808 18:20:25.204872 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:25.204889 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:25.204932 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:20:25.205781 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:25.205789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:25.206201 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:25.206513 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:25.208337 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.209841 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.211800 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.212499 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.212673 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:25.215976 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:25.215980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:25.215990 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:25.216015 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:25.217556 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:25.218092 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:25.218097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:25.218101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:25.218206 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:25.218566 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:25.218573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:25.218574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:25.218938 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:25.219097 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:25.219650 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 18:20:25.219891 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":142,"threads_started":1}
I20260808 18:20:25.220115 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.204510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:35479 (local address 127.27.18.195:46099)
0808 18:20:25.204872 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:25.204889 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:25.204932 (+ 43us) server_negotiation.cc:407] Connection header received
0808 18:20:25.205781 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:25.205788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:25.206201 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:25.206513 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:25.208338 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.209841 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.211800 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.212499 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.212673 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:25.215652 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:25.215657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:25.215668 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:25.215693 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:25.217507 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:25.218061 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:25.218069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:25.218079 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:25.218188 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:25.218558 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:25.218561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:25.218563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:25.218939 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:25.219097 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:25.219677 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 18:20:25.219934 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":104,"threads_started":1}
I20260808 18:20:25.220127 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.204193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:46099 (local address 127.27.18.193:35479)
0808 18:20:25.204449 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:25.204464 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:25.204604 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:25.206604 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:25.206610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:25.207272 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:25.208146 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:25.208167 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.209988 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.209991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:25.211666 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:25.211677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:25.212636 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:25.212640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:25.212997 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:25.213713 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:25.213734 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:25.215458 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:25.217636 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:25.217642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:25.217654 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:25.217938 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:25.218287 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:25.218292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:25.218294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:25.218462 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:25.219062 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:25.219080 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:25.219314 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 18:20:25.219541 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 18:20:25.219934 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":539,"spinlock_wait_cycles":23552,"thread_start_us":60,"threads_started":1}
W20260808 18:20:25.222743 18842 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
W20260808 18:20:25.222931 18838 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
I20260808 18:20:25.223049 18838 leader_election.cc:302] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59de6ece509c45e6ac435edc7178fc53; no voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9
I20260808 18:20:25.223312 19430 raft_consensus.cc:2726] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:20:25.397277 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.397021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:46635)
0808 18:20:25.397142 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:25.397192 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 18:20:25.528403 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.527936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:35113)
0808 18:20:25.528213 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:25.528272 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
I20260808 18:20:25.700281 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:25.699878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:51761)
0808 18:20:25.700121 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:25.700171 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
I20260808 18:20:26.398407 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:26.398002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:46999)
0808 18:20:26.398244 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:26.398297 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
I20260808 18:20:26.529539 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:26.529190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:60155)
0808 18:20:26.529418 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:26.529464 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
I20260808 18:20:26.701476 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:26.701106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:35331)
0808 18:20:26.701342 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:26.701393 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":74,"threads_started":1}
I20260808 18:20:27.399667 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:27.399119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:39875)
0808 18:20:27.399455 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:27.399521 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":72,"threads_started":1}
I20260808 18:20:27.453405 19449 raft_consensus.cc:482] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:27.453815 19449 raft_consensus.cc:504] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:27.454394 19449 leader_election.cc:288] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099), 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
W20260808 18:20:27.455096 18838 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
W20260808 18:20:27.455240 18842 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
I20260808 18:20:27.455389 18842 leader_election.cc:302] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59de6ece509c45e6ac435edc7178fc53; no voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9
I20260808 18:20:27.455597 19449 raft_consensus.cc:2726] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:20:27.530658 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:27.530257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:33751)
0808 18:20:27.530501 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:27.530549 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
I20260808 18:20:27.702685 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:27.702317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:38757)
0808 18:20:27.702561 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:27.702609 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260808 18:20:28.400892 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:28.400448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:48591)
0808 18:20:28.400736 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:28.400787 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":72,"threads_started":1}
I20260808 18:20:28.531942 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:28.531518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:49363)
0808 18:20:28.531761 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:28.531809 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260808 18:20:28.703742 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:28.703346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:34811)
0808 18:20:28.703590 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:28.703640 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
I20260808 18:20:29.402191 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:29.401696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:36475)
0808 18:20:29.402036 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:29.402099 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
I20260808 18:20:29.533059 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:29.532581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:36327)
0808 18:20:29.532919 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:29.532969 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":86,"threads_started":1}
I20260808 18:20:29.704891 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:29.704479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:34139)
0808 18:20:29.704746 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:29.704806 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
I20260808 18:20:30.284240 19464 raft_consensus.cc:482] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:30.284422 19464 raft_consensus.cc:504] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:30.284787 19464 leader_election.cc:288] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099), 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
W20260808 18:20:30.285552 18838 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
W20260808 18:20:30.285722 18842 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
I20260808 18:20:30.285807 18842 leader_election.cc:302] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59de6ece509c45e6ac435edc7178fc53; no voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9
I20260808 18:20:30.285940 19464 raft_consensus.cc:2726] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:20:30.403291 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:30.402932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:45653)
0808 18:20:30.403153 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:30.403199 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260808 18:20:30.534353 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:30.533932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:34977)
0808 18:20:30.534205 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:30.534254 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":70,"threads_started":1}
I20260808 18:20:30.705993 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:30.705534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:51157)
0808 18:20:30.705856 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:30.705908 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":68,"threads_started":1}
I20260808 18:20:31.404476 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:31.404071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:53683)
0808 18:20:31.404315 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:31.404381 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260808 18:20:31.535547 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:31.535153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:40083)
0808 18:20:31.535400 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:31.535459 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":68,"threads_started":1}
I20260808 18:20:31.707134 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:31.706741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:59817)
0808 18:20:31.707001 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:31.707052 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260808 18:20:32.405568 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:32.405192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:32921)
0808 18:20:32.405425 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:32.405476 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260808 18:20:32.536537 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:32.536162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:46599)
0808 18:20:32.536399 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:32.536449 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":70,"threads_started":1}
I20260808 18:20:32.708166 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:32.707756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:48047)
0808 18:20:32.708028 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:32.708079 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":88,"threads_started":1}
I20260808 18:20:33.091285 19478 raft_consensus.cc:482] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:33.091467 19478 raft_consensus.cc:504] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b46a67ee04915973f862ef2e597f9" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 46099 } } peers { permanent_uuid: "71173bf6e0a24e18ae02154f77fa451b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33373 } } peers { permanent_uuid: "59de6ece509c45e6ac435edc7178fc53" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 35539 } }
I20260808 18:20:33.091822 19478 leader_election.cc:288] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099), 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373)
W20260808 18:20:33.092506 18838 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b29b46a67ee04915973f862ef2e597f9 (127.27.18.195:46099): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
W20260808 18:20:33.092592 18842 leader_election.cc:341] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 71173bf6e0a24e18ae02154f77fa451b (127.27.18.194:33373): Not found: Tablet not found: 7abadb447305465597bda98b49b647ad
I20260808 18:20:33.092711 18842 leader_election.cc:302] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59de6ece509c45e6ac435edc7178fc53; no voters: 71173bf6e0a24e18ae02154f77fa451b, b29b46a67ee04915973f862ef2e597f9
I20260808 18:20:33.092862 19478 raft_consensus.cc:2726] T 7abadb447305465597bda98b49b647ad P 59de6ece509c45e6ac435edc7178fc53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:20:33.406898 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:33.406465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:38955)
0808 18:20:33.406725 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:33.406800 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260808 18:20:33.537724 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:33.537232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:53703)
0808 18:20:33.537578 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:33.537630 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":88,"threads_started":1}
I20260808 18:20:33.709216 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:33.708846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:45257)
0808 18:20:33.709078 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:33.709128 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260808 18:20:34.408039 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:34.407647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:47449)
0808 18:20:34.407896 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:34.407946 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260808 18:20:34.538869 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:34.538479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:46911)
0808 18:20:34.538712 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:34.538764 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 18:20:34.710498 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:34.710025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:55243)
0808 18:20:34.710280 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:34.710380 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260808 18:20:35.409191 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:35.408709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:57459)
0808 18:20:35.409045 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:35.409098 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":80,"threads_started":1}
I20260808 18:20:35.540163 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:35.539754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:58727)
0808 18:20:35.540011 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:35.540061 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":103,"threads_started":1}
I20260808 18:20:35.711629 19494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:35.711190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:38957)
0808 18:20:35.711501 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:35.711550 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":119,"threads_started":1}
I20260808 18:20:36.410341 19498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:36.409937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:54989)
0808 18:20:36.410189 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:36.410236 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260808 18:20:36.541339 19499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:36.540939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:39337)
0808 18:20:36.541185 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:36.541236 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
I20260808 18:20:36.712637 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:36.712270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:47525)
0808 18:20:36.712500 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:36.712551 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260808 18:20:37.411876 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:37.411293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:54855)
0808 18:20:37.411622 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:37.411742 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":133,"threads_started":1}
I20260808 18:20:37.542469 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:37.542086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:60455)
0808 18:20:37.542330 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:37.542378 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260808 18:20:37.713722 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:37.713342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:47119)
0808 18:20:37.713583 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:37.713632 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260808 18:20:38.413125 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:38.412744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.193:48667)
0808 18:20:38.412986 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:38.413034 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260808 18:20:38.543718 19507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:38.543249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.194:38077)
0808 18:20:38.543573 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:38.543626 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":147,"threads_started":1}
I20260808 18:20:38.705066 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 18761
I20260808 18:20:38.714861 19508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:38.714501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38431 (local address 127.27.18.195:57941)
0808 18:20:38.714726 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:38.714772 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":83,"threads_started":1}
I20260808 18:20:38.718886 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 19017
I20260808 18:20:38.731137 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 19212
2026-08-08T18:20:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19194 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:20:38 dist-test-slave-x4qh krb5kdc[19511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:20:38 dist-test-slave-x4qh krb5kdc[19511](info): set up 2 sockets
Aug 08 18:20:38 dist-test-slave-x4qh krb5kdc[19511](info): commencing operation
krb5kdc: starting...
W20260808 18:20:39.910194 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:20:39 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:20:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:20:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:20:40.030606 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33333
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:40.132584 19536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:40.132822 19536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:40.132897 19536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:40.136591 19536 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:20:40.136650 19536 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:20:40.136674 19536 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:20:40.136709 19536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:20:40.136749 19536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:40.136786 19536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:40.136844 19536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:20:40.136880 19536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:20:40.141484 19536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:40.142309 19536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:40.143179 19536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:40.151031 19546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.151017 19543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.151017 19542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:40.151086 19536 server_base.cc:1034] running on GCE node
I20260808 18:20:40.151808 19536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:40.152697 19536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:40.153874 19536 hybrid_clock.cc:648] HybridClock initialized: now 1786213240153854 us; error 37 us; skew 500 ppm
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:40.156338 19536 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:20:40.156993 19536 webserver.cc:460] Webserver started at http://127.27.18.254:45695/ using document root <none> and password file <none>
I20260808 18:20:40.157428 19536 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:40.157476 19536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:40.157629 19536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:40.159458 19536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "375e8dc5e78547c18dd16755daecd7bb"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "dc935ddaeff3092300eb6781d06ff905"
server_key_iv: "f5e11941a9133ffec44514b4cf83c3d4"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.159900 19536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "375e8dc5e78547c18dd16755daecd7bb"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "dc935ddaeff3092300eb6781d06ff905"
server_key_iv: "f5e11941a9133ffec44514b4cf83c3d4"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.163089 19536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:20:40.165243 19553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.166283 19536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:40.166391 19536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "375e8dc5e78547c18dd16755daecd7bb"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "dc935ddaeff3092300eb6781d06ff905"
server_key_iv: "f5e11941a9133ffec44514b4cf83c3d4"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.166463 19536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:40.178275 19536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:40.180598 19536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:40.180722 19536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:40.186883 19536 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:33333
I20260808 18:20:40.187121 19634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:33333 every 8 connection(s)
I20260808 18:20:40.187705 19536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:20:40.190125 19637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:40.194846 19637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap starting.
I20260808 18:20:40.196417 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 19536
I20260808 18:20:40.196569 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:20:40.196889 27723 external_mini_cluster.cc:1428] Setting key f6b977f0c5d923092ac14dabfa45d32f
I20260808 18:20:40.197460 19637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:40.198347 19637 log.cc:824] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:40.200314 19637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: No bootstrap required, opened a new log
I20260808 18:20:40.204160 19637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:20:40.204468 19637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:40.204536 19637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375e8dc5e78547c18dd16755daecd7bb, State: Initialized, Role: FOLLOWER
I20260808 18:20:40.205025 19637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:20:40.205183 19637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:20:40.205248 19637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:20:40.205349 19637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 0 FOLLOWER]: Advancing to term 1
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:40.206314 19637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:20:40.206779 19637 leader_election.cc:302] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375e8dc5e78547c18dd16755daecd7bb; no voters:
I20260808 18:20:40.207151 19637 leader_election.cc:288] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:20:40.207355 19647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:40.207631 19647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 LEADER]: Becoming Leader. State: Replica: 375e8dc5e78547c18dd16755daecd7bb, State: Running, Role: LEADER
I20260808 18:20:40.207957 19647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:20:40.208448 19637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:20:40.209580 19651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "375e8dc5e78547c18dd16755daecd7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } } }
I20260808 18:20:40.209719 19651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:40.210038 19653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375e8dc5e78547c18dd16755daecd7bb. Latest consensus state: current_term: 1 leader_uuid: "375e8dc5e78547c18dd16755daecd7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } } }
I20260808 18:20:40.210197 19653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: This master's current role is: LEADER
I20260808 18:20:40.211050 19665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:20:40.211848 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.197967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.27.18.254:33333)
0808 18:20:40.198342 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.198356 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.198389 (+ 33us) server_negotiation.cc:407] Connection header received
0808 18:20:40.199209 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.199229 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.199584 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.199819 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:40.200652 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.201497 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.202583 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.202980 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.203083 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.206085 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:40.206091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:40.206107 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:40.206143 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:40.208182 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.208667 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.208672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.208678 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.208817 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.209134 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.209139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.209144 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.209750 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:40.209941 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.210739 (+ 798us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.211043 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":117,"threads_started":1}
I20260808 18:20:40.214120 19665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:20:40.219230 19665 catalog_manager.cc:1347] Generated new cluster ID: 6c7dcd4a2f374a43a1a78a0b168b5562
I20260808 18:20:40.219301 19665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:20:40.231330 19665 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:20:40.232406 19665 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:20:40.243326 19665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Generated new TSK 0
I20260808 18:20:40.244035 19665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:20:40.275991 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33333
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:40.383461 19683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:40.383730 19683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:40.383809 19683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:40.387648 19683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:40.387729 19683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:40.387820 19683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:20:40.392526 19683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:40.393409 19683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:40.394318 19683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:40.403092 19695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.403101 19689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:40.403160 19683 server_base.cc:1034] running on GCE node
W20260808 18:20:40.403185 19691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:40.403755 19683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:40.404350 19683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:40.405561 19683 hybrid_clock.cc:648] HybridClock initialized: now 1786213240405531 us; error 42 us; skew 500 ppm
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:40.408079 19683 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:20:40.408927 19683 webserver.cc:460] Webserver started at http://127.27.18.193:38653/ using document root <none> and password file <none>
I20260808 18:20:40.409487 19683 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:40.409569 19683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:40.409785 19683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:40.411688 19683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "27c759855f9344ea9a0ce570fef54661"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "77114a726bd3df0c913d33e6b444978d"
server_key_iv: "d6dbb016833e7266f1cd6bc21800f878"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.412227 19683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "27c759855f9344ea9a0ce570fef54661"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "77114a726bd3df0c913d33e6b444978d"
server_key_iv: "d6dbb016833e7266f1cd6bc21800f878"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.415606 19683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:20:40.417738 19710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.418902 19683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:20:40.419034 19683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "27c759855f9344ea9a0ce570fef54661"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "77114a726bd3df0c913d33e6b444978d"
server_key_iv: "d6dbb016833e7266f1cd6bc21800f878"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.419138 19683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:40.436745 19683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:40.439203 19683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:40.439441 19683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:40.440008 19683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:40.440737 19683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:40.440814 19683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.440903 19683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:40.440953 19683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.449942 19683 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:43263
I20260808 18:20:40.449976 19890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:43263 every 8 connection(s)
I20260808 18:20:40.450731 19683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:20:40.451864 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 19683
I20260808 18:20:40.451993 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:20:40.452272 27723 external_mini_cluster.cc:1428] Setting key 5d3b605841f9f526bb1719cc9e6ebda7
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:20:40.464856 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.452453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36473 (local address 127.27.18.254:33333)
0808 18:20:40.452589 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.452594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.453389 (+ 795us) server_negotiation.cc:407] Connection header received
0808 18:20:40.454530 (+ 1141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.454536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.454594 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.454681 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:40.455847 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.456328 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.457929 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.458152 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.458233 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.461314 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:40.461317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:40.461319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:40.461341 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:40.462671 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.463252 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.463254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.463256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.463470 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.463833 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.463836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.463838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.464006 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:40.464111 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.464578 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.464695 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:20:40.465521 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.452817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:36473)
0808 18:20:40.453241 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:40.453285 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:40.454339 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:40.454803 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:40.454815 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:40.455118 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:40.455689 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.455710 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.456446 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.456451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.457757 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.457771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.458308 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.458314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.458584 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.459155 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:40.459176 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:40.461146 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:40.462796 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:40.462815 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:40.462830 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:40.463151 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:40.463580 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:40.463583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:40.463585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:40.463732 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:40.464127 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:40.464134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:40.464355 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 18:20:40.464535 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 18:20:40.464827 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:20:40.466665 19893 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33333
I20260808 18:20:40.466907 19893 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:40.467485 19893 heartbeater.cc:511] Master 127.27.18.254:33333 requested a full tablet report, sending...
I20260808 18:20:40.468986 19578 ts_manager.cc:194] Registered new tserver with Master: 27c759855f9344ea9a0ce570fef54661 (127.27.18.193:43263)
I20260808 18:20:40.470155 19578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:36473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:20:40.482436 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33333
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:40.586903 19902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:40.587143 19902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:40.587208 19902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:40.590950 19902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:40.591024 19902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:40.591111 19902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:20:40.595763 19902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:40.596658 19902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:40.597553 19902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:40.606581 19908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.606554 19907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.606571 19911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:40.606953 19902 server_base.cc:1034] running on GCE node
I20260808 18:20:40.607336 19902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:40.607908 19902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:40.609102 19902 hybrid_clock.cc:648] HybridClock initialized: now 1786213240609081 us; error 29 us; skew 500 ppm
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:40.611616 19902 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:20:40.612468 19902 webserver.cc:460] Webserver started at http://127.27.18.194:40917/ using document root <none> and password file <none>
I20260808 18:20:40.613004 19902 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:40.613122 19902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:40.613358 19902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:40.615170 19902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "8e1dd3d029d8413591c4f504eb7f4167"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "8eaa9274dea41897512aa59bf423cf0b"
server_key_iv: "544f6067a998de8a678c38098d879813"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.615721 19902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "8e1dd3d029d8413591c4f504eb7f4167"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "8eaa9274dea41897512aa59bf423cf0b"
server_key_iv: "544f6067a998de8a678c38098d879813"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.618999 19902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:20:40.621304 19923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.622412 19902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:20:40.622538 19902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "8e1dd3d029d8413591c4f504eb7f4167"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "8eaa9274dea41897512aa59bf423cf0b"
server_key_iv: "544f6067a998de8a678c38098d879813"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.622677 19902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:40.635077 19902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:40.637547 19902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:40.637740 19902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:40.638300 19902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:40.639060 19902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:40.639128 19902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.639194 19902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:40.639245 19902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.647954 19902 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:43419
I20260808 18:20:40.647982 20111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:43419 every 8 connection(s)
I20260808 18:20:40.648758 19902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:40.658428 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 19902
I20260808 18:20:40.658582 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:20:40.658914 27723 external_mini_cluster.cc:1428] Setting key a480b85ef48e32bd7b008fb1de09e521
I20260808 18:20:40.662143 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.650521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:49691 (local address 127.27.18.254:33333)
0808 18:20:40.650648 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.650653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.651358 (+ 705us) server_negotiation.cc:407] Connection header received
0808 18:20:40.652592 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.652596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.652653 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.652758 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:40.654062 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.654766 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.655876 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.656106 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.656223 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.658567 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:40.658571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:40.658574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:40.658609 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:40.660218 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.660684 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.660688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.660689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.660781 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.661096 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.661099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.661100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.661276 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:40.661396 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.661862 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.661988 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:20:40.662982 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.650813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:49691)
0808 18:20:40.651219 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:40.651257 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:40.652388 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:40.652922 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:40.652928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:40.653248 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:40.653878 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.653895 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.654891 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.654895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.655742 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.655752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.656263 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.656268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.656443 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.657005 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:40.657020 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:40.658395 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:40.660327 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:40.660332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:40.660345 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:40.660577 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:40.660871 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:40.660874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:40.660876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:40.661004 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:40.661397 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:40.661404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:40.661628 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 18:20:40.661817 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 18:20:40.662111 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
I20260808 18:20:40.664101 20112 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33333
I20260808 18:20:40.664361 20112 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:40.664850 20112 heartbeater.cc:511] Master 127.27.18.254:33333 requested a full tablet report, sending...
I20260808 18:20:40.665861 19578 ts_manager.cc:194] Registered new tserver with Master: 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419)
I20260808 18:20:40.666374 19578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:49691
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:20:40.685927 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33333
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:20:40.789915 20118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:20:40.790114 20118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:20:40.790149 20118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:20:40.793784 20118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:20:40.793834 20118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:20:40.793883 20118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:20:40.798580 20118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:20:40.799539 20118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:20:40.800455 20118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:20:40.809538 20125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.809624 20127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:20:40.809624 20124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:20:40.809890 20118 server_base.cc:1034] running on GCE node
I20260808 18:20:40.810303 20118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:20:40.810927 20118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:20:40.812109 20118 hybrid_clock.cc:648] HybridClock initialized: now 1786213240812083 us; error 37 us; skew 500 ppm
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:20:40.814653 20118 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:20:40.815466 20118 webserver.cc:460] Webserver started at http://127.27.18.195:36275/ using document root <none> and password file <none>
I20260808 18:20:40.815960 20118 fs_manager.cc:359] Metadata directory not provided
I20260808 18:20:40.816018 20118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:20:40.816252 20118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:20:40.818116 20118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "aededf31d0b94833bf11a929c29dcccf"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "4d986d55128328aac963ab688c216bdc"
server_key_iv: "e143a7109d3fb8816a574f55a839137d"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.818642 20118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "aededf31d0b94833bf11a929c29dcccf"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "4d986d55128328aac963ab688c216bdc"
server_key_iv: "e143a7109d3fb8816a574f55a839137d"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.822257 20118 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:20:40.824681 20139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.825870 20118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:20:40.825985 20118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "aededf31d0b94833bf11a929c29dcccf"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "4d986d55128328aac963ab688c216bdc"
server_key_iv: "e143a7109d3fb8816a574f55a839137d"
server_key_version: "encryptionkey@0"
I20260808 18:20:40.826083 20118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:20:40.852321 20118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:20:40.854892 20118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:20:40.855098 20118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:20:40.855693 20118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:20:40.856528 20118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:20:40.856576 20118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.856647 20118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:20:40.856705 20118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:20:40.865445 20118 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:39453
I20260808 18:20:40.865471 20324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:39453 every 8 connection(s)
I20260808 18:20:40.866277 20118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:20:40.871610 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 20118
I20260808 18:20:40.871732 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:20:40.871995 27723 external_mini_cluster.cc:1428] Setting key 67b2477f38a90280e3498142a60b41f6
Aug 08 18:20:40 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213240, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:20:40.878492 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.867918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:55243 (local address 127.27.18.254:33333)
0808 18:20:40.868055 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.868058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.868803 (+ 745us) server_negotiation.cc:407] Connection header received
0808 18:20:40.870062 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.870065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.870123 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.870235 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:40.871238 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.871974 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.873150 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.873410 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.873505 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.875757 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:40.875760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:40.875762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:40.875791 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:40.876891 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.877302 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.877304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.877305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.877373 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.877666 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.877668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.877669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.877822 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:40.877895 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.878294 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.878383 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:20:40.879168 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.868248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:55243)
0808 18:20:40.868657 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:40.868694 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:40.869796 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:40.870392 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:40.870397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:40.870693 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:20:40.871097 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.871109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.872139 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.872142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.873047 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.873057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.873543 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.873545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.873704 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.874212 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:20:40.874228 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:20:40.875619 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:20:40.876988 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:40.876992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:40.877002 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:40.877219 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:40.877448 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:20:40.877450 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:20:40.877452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:20:40.877591 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:20:40.877906 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:20:40.877910 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:20:40.878090 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 18:20:40.878242 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 18:20:40.878452 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260808 18:20:40.880185 20326 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33333
I20260808 18:20:40.880391 20326 heartbeater.cc:464] Registering TS with master...
I20260808 18:20:40.880786 20326 heartbeater.cc:511] Master 127.27.18.254:33333 requested a full tablet report, sending...
I20260808 18:20:40.881639 19578 ts_manager.cc:194] Registered new tserver with Master: aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
I20260808 18:20:40.882169 19578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:55243
I20260808 18:20:40.885715 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:20:40.893929 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.887455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.27.18.254:33333)
0808 18:20:40.887557 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.887561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.887721 (+ 160us) server_negotiation.cc:407] Connection header received
0808 18:20:40.887814 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.887816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.887850 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.887912 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:20:40.888501 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.888968 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.889712 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.889912 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.889985 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.891180 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:20:40.891186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:20:40.891191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:20:40.891212 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:20:40.892638 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.893009 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.893011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.893013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.893081 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:20:40.893299 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:20:40.893301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:20:40.893302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:20:40.893474 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:20:40.893571 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.893755 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.893840 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:20:40.896257 19578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:20:40.899030 19578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:20:40.912842 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.906923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43419 (local address 127.0.0.1:50812)
0808 18:20:40.907331 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:40.907416 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:40.907542 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:40.908437 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:40.908445 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:40.908472 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:40.908603 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.908610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.910037 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.910041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.911189 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.911200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.912292 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.912295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.912424 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.912465 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:20:40.912574 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 18:20:40.912693 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260808 18:20:40.912889 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.907543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.27.18.194:43419)
0808 18:20:40.907917 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.907930 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.907955 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:20:40.908028 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.908035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.908255 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.908745 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:40.908920 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.909917 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.911345 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.912167 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.912274 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.912403 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.912623 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.912710 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":93,"threads_started":1}
I20260808 18:20:40.913358 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.907571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:39453 (local address 127.0.0.1:48936)
0808 18:20:40.908431 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:40.908442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:40.908507 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:40.908985 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:40.908991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:40.909015 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:40.909117 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.909126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.910582 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.910585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.911805 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.911815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.912928 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.912932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.913046 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.913067 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:20:40.913150 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:20:40.913219 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":76,"thread_start_us":88,"threads_started":1}
I20260808 18:20:40.913568 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.907057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43263 (local address 127.0.0.1:59324)
0808 18:20:40.908134 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:40.908147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:40.908243 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:40.909147 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:40.909150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:40.909169 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:40.909249 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.909255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.910570 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.910573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.911824 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:40.911835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.913088 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.913094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:40.913218 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.913247 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:20:40.913343 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 18:20:40.913426 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":82,"thread_start_us":85,"threads_started":1}
I20260808 18:20:40.913802 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.907731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.27.18.195:39453)
0808 18:20:40.908455 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.908468 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.908492 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:20:40.908652 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.908657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.908840 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.908987 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:40.909267 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.910425 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.911916 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.912600 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.912736 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.912859 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.913577 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.913678 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":144,"threads_started":1}
I20260808 18:20:40.914093 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:40.907796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.27.18.193:43263)
0808 18:20:40.908232 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:40.908244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:40.908530 (+ 286us) server_negotiation.cc:407] Connection header received
0808 18:20:40.908616 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:40.908620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:40.908913 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:40.909085 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:40.909419 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.910446 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.911959 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:40.912776 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:40.912877 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:40.913021 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:40.913827 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 18:20:40.913934 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":120,"threads_started":1}
I20260808 18:20:40.915779 20000 tablet_service.cc:1467] Processing CreateTablet for tablet 1db99b6b20674737adfdbf0ff6569abb (DEFAULT_TABLE table=test-table [id=1c3d1c635d194f05b1edde214abf5ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:40.916112 19789 tablet_service.cc:1467] Processing CreateTablet for tablet 1db99b6b20674737adfdbf0ff6569abb (DEFAULT_TABLE table=test-table [id=1c3d1c635d194f05b1edde214abf5ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:40.916697 20000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db99b6b20674737adfdbf0ff6569abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:40.916965 19789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db99b6b20674737adfdbf0ff6569abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:40.922489 20345 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Bootstrap starting.
I20260808 18:20:40.923275 20346 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Bootstrap starting.
I20260808 18:20:40.924718 20345 tablet_bootstrap.cc:654] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:40.924823 20222 tablet_service.cc:1467] Processing CreateTablet for tablet 1db99b6b20674737adfdbf0ff6569abb (DEFAULT_TABLE table=test-table [id=1c3d1c635d194f05b1edde214abf5ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:20:40.925763 20346 tablet_bootstrap.cc:654] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:40.925779 20345 log.cc:824] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:40.925865 20222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db99b6b20674737adfdbf0ff6569abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:20:40.926550 20346 log.cc:824] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:40.927712 20345 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: No bootstrap required, opened a new log
I20260808 18:20:40.927909 20345 ts_tablet_manager.cc:1397] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:20:40.928364 20346 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: No bootstrap required, opened a new log
I20260808 18:20:40.928539 20346 ts_tablet_manager.cc:1397] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:20:40.931088 20345 raft_consensus.cc:348] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:40.931295 20345 raft_consensus.cc:374] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:40.931408 20345 raft_consensus.cc:729] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c759855f9344ea9a0ce570fef54661, State: Initialized, Role: FOLLOWER
I20260808 18:20:40.931815 20345 consensus_queue.cc:260] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:40.932044 20346 raft_consensus.cc:348] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:40.932338 20346 raft_consensus.cc:374] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:40.932387 19893 heartbeater.cc:503] Master 127.27.18.254:33333 was elected leader, sending a full tablet report...
I20260808 18:20:40.932422 20346 raft_consensus.cc:729] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e1dd3d029d8413591c4f504eb7f4167, State: Initialized, Role: FOLLOWER
I20260808 18:20:40.932694 20345 ts_tablet_manager.cc:1428] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:20:40.933043 20346 consensus_queue.cc:260] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:40.933239 20353 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Bootstrap starting.
I20260808 18:20:40.933792 20112 heartbeater.cc:503] Master 127.27.18.254:33333 was elected leader, sending a full tablet report...
I20260808 18:20:40.933898 20346 ts_tablet_manager.cc:1428] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:20:40.935660 20353 tablet_bootstrap.cc:654] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Neither blocks nor log segments found. Creating new log.
I20260808 18:20:40.936427 20353 log.cc:824] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Log is configured to *not* fsync() on all Append() calls
I20260808 18:20:40.937928 20353 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: No bootstrap required, opened a new log
I20260808 18:20:40.938119 20353 ts_tablet_manager.cc:1397] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:20:40.940608 20353 raft_consensus.cc:348] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:40.940811 20353 raft_consensus.cc:374] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:20:40.940864 20353 raft_consensus.cc:729] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aededf31d0b94833bf11a929c29dcccf, State: Initialized, Role: FOLLOWER
I20260808 18:20:40.941172 20353 consensus_queue.cc:260] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:40.941967 20353 ts_tablet_manager.cc:1428] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 18:20:40.941968 20326 heartbeater.cc:503] Master 127.27.18.254:33333 was elected leader, sending a full tablet report...
W20260808 18:20:40.951819 19894 tablet.cc:2369] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:20:41.117242 20327 tablet.cc:2369] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:20:41.150023 20114 tablet.cc:2369] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:20:41.240806 20351 raft_consensus.cc:482] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:20:41.241019 20351 raft_consensus.cc:504] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:41.242250 20351 leader_election.cc:288] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
I20260808 18:20:41.246865 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.242432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43419 (local address 127.27.18.193:36923)
0808 18:20:41.242790 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:41.242817 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:41.242987 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:41.243273 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:41.243276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:41.243293 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:41.243452 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:41.243459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.244302 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.244305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:41.245307 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:41.245319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.246383 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.246386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:41.246485 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:41.246511 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:20:41.246587 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:20:41.246668 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"spinlock_wait_cycles":11136,"thread_start_us":99,"threads_started":1}
I20260808 18:20:41.246913 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.242609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36923 (local address 127.27.18.194:43419)
0808 18:20:41.242749 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:41.242753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:41.242886 (+ 133us) server_negotiation.cc:407] Connection header received
0808 18:20:41.243127 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:41.243130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:41.243174 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:41.243260 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:41.243619 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.244185 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.245452 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.246229 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.246386 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:41.246467 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:41.246633 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 18:20:41.246729 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:20:41.247010 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.243067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:39557 (local address 127.27.18.195:39453)
0808 18:20:41.243214 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:41.243219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:41.243237 (+ 18us) server_negotiation.cc:407] Connection header received
0808 18:20:41.243302 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:41.243307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:41.243409 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:41.243498 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:20:41.243801 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.244684 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.245873 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.246275 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.246413 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:41.246467 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:41.246766 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 18:20:41.246872 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:20:41.247458 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.242563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:39453 (local address 127.27.18.193:39557)
0808 18:20:41.242877 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:41.242889 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:20:41.242987 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:20:41.243537 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:20:41.243543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:20:41.243565 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:20:41.243680 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:41.243687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.244825 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.244829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:41.245744 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:20:41.245753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.246519 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.246522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:20:41.246641 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:41.246658 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:20:41.247180 (+ 522us) client_negotiation.cc:240] Negotiation successful
0808 18:20:41.247264 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"spinlock_wait_cycles":23680,"thread_start_us":127,"threads_started":1}
I20260808 18:20:41.247879 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db99b6b20674737adfdbf0ff6569abb" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1dd3d029d8413591c4f504eb7f4167" is_pre_election: true
I20260808 18:20:41.248199 20029 raft_consensus.cc:2454] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 0.
I20260808 18:20:41.248200 20254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db99b6b20674737adfdbf0ff6569abb" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aededf31d0b94833bf11a929c29dcccf" is_pre_election: true
I20260808 18:20:41.248526 20254 raft_consensus.cc:2454] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 0.
I20260808 18:20:41.248742 19712 leader_election.cc:302] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661, 8e1dd3d029d8413591c4f504eb7f4167; no voters:
I20260808 18:20:41.249054 20351 raft_consensus.cc:2781] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:20:41.249130 20351 raft_consensus.cc:482] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:20:41.249171 20351 raft_consensus.cc:3037] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:41.250154 20351 raft_consensus.cc:504] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:41.250540 20351 leader_election.cc:288] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 election: Requested vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
I20260808 18:20:41.250842 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db99b6b20674737adfdbf0ff6569abb" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1dd3d029d8413591c4f504eb7f4167"
I20260808 18:20:41.250952 20254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db99b6b20674737adfdbf0ff6569abb" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aededf31d0b94833bf11a929c29dcccf"
I20260808 18:20:41.250978 20029 raft_consensus.cc:3037] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:41.251067 20254 raft_consensus.cc:3037] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:20:41.252262 20254 raft_consensus.cc:2454] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 1.
I20260808 18:20:41.252262 20029 raft_consensus.cc:2454] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 1.
I20260808 18:20:41.252576 19712 leader_election.cc:302] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661, 8e1dd3d029d8413591c4f504eb7f4167; no voters:
I20260808 18:20:41.252777 20351 raft_consensus.cc:2781] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:20:41.253010 20351 raft_consensus.cc:686] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 LEADER]: Becoming Leader. State: Replica: 27c759855f9344ea9a0ce570fef54661, State: Running, Role: LEADER
I20260808 18:20:41.253417 20351 consensus_queue.cc:237] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:20:41.257364 19575 catalog_manager.cc:5495] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27c759855f9344ea9a0ce570fef54661 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "27c759855f9344ea9a0ce570fef54661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } health_report { overall_health: UNKNOWN } } }
I20260808 18:20:41.273149 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.269724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.27.18.193:43263)
0808 18:20:41.269842 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 18:20:41.269847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:20:41.269918 (+ 71us) server_negotiation.cc:407] Connection header received
0808 18:20:41.270086 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:20:41.270089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:20:41.270147 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:20:41.270256 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:20:41.270492 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.270995 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.271843 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:20:41.272053 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:20:41.272151 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:20:41.272441 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:20:41.272803 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:20:41.272931 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:20:41.272990 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 18:20:41.273038 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:20:41.279054 20254 raft_consensus.cc:1261] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Refusing update from remote peer 27c759855f9344ea9a0ce570fef54661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:41.279215 20029 raft_consensus.cc:1261] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Refusing update from remote peer 27c759855f9344ea9a0ce570fef54661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:20:41.279740 20362 consensus_queue.cc:1035] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:41.279938 20351 consensus_queue.cc:1035] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:20:41.294394 20368 mvcc.cc:205] Tried to move back new op lower bound from 7316329436272070656 to 7316329436177047552. Current Snapshot: MvccSnapshot[applied={T|T < 7316329436272070656}]
I20260808 18:20:41.298064 19578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:41.298241 19578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:20:41.300406 19578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Sending DeleteTablet for 3 replicas of tablet 1db99b6b20674737adfdbf0ff6569abb
I20260808 18:20:41.301180 19789 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db99b6b20674737adfdbf0ff6569abb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:41 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:59324
I20260808 18:20:41.301255 20222 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db99b6b20674737adfdbf0ff6569abb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:41 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:48936
I20260808 18:20:41.301767 20000 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db99b6b20674737adfdbf0ff6569abb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:41 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50812
I20260808 18:20:41.301937 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 19683
I20260808 18:20:41.310945 20379 tablet_replica.cc:330] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: stopping tablet replica
I20260808 18:20:41.311282 20379 raft_consensus.cc:2229] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:41.311628 20379 raft_consensus.cc:2258] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:41.313110 20379 ts_tablet_manager.cc:1905] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:41.315641 20380 tablet_replica.cc:330] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: stopping tablet replica
I20260808 18:20:41.316085 20380 raft_consensus.cc:2229] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:20:41.316556 20380 raft_consensus.cc:2258] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:20:41.316599 20379 ts_tablet_manager.cc:1918] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:20:41.316787 20379 log.cc:1196] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/1db99b6b20674737adfdbf0ff6569abb
I20260808 18:20:41.317286 20379 ts_tablet_manager.cc:1939] T 1db99b6b20674737adfdbf0ff6569abb P aededf31d0b94833bf11a929c29dcccf: Deleting consensus metadata
I20260808 18:20:41.318480 20380 ts_tablet_manager.cc:1905] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:20:41.318665 19558 catalog_manager.cc:4841] TS aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453): tablet 1db99b6b20674737adfdbf0ff6569abb (table test-table [id=1c3d1c635d194f05b1edde214abf5ff2]) successfully deleted
I20260808 18:20:41.320354 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 19902
W20260808 18:20:41.323688 19558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:43263 (error 108)
I20260808 18:20:41.324891 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.324527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43263 (local address 127.0.0.1:59336)
0808 18:20:41.324694 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:41.324781 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:43263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:20:41.325148 19558 catalog_manager.cc:4584] TS 27c759855f9344ea9a0ce570fef54661 (127.27.18.193:43263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1db99b6b20674737adfdbf0ff6569abb: Network error: Client connection negotiation failed: client connection to 127.27.18.193:43263: connect: Connection refused (error 111)
I20260808 18:20:41.329681 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 20118
I20260808 18:20:41.329864 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:20:41.329547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43419 (local address 127.0.0.1:50814)
0808 18:20:41.329730 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 18:20:41.329784 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.194:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 18:20:41.339664 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 19536
I20260808 18:21:01.350924 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33333
--webserver_interface=127.27.18.254
--webserver_port=45695
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:01.458241 20387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:01.458447 20387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:01.458513 20387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:01.462460 20387 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:21:01.462503 20387 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:21:01.462518 20387 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:21:01.462553 20387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:01.462587 20387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:01.462625 20387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:01.462658 20387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:01.462672 20387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:01.467617 20387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=45695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:01.468479 20387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:01.469352 20387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:01.482961 20400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:01.482963 20396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:01.482963 20398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:01.483017 20387 server_base.cc:1034] running on GCE node
I20260808 18:21:01.483803 20387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:01.484784 20387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:01.485953 20387 hybrid_clock.cc:648] HybridClock initialized: now 1786213261485940 us; error 24 us; skew 500 ppm
Aug 08 18:21:01 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213261, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:01.491662 20387 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:21:01.492439 20387 webserver.cc:460] Webserver started at http://127.27.18.254:45695/ using document root <none> and password file <none>
I20260808 18:21:01.492947 20387 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:01.493021 20387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:01.496400 20387 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 18:21:01.498239 20409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:01.499294 20387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:21:01.499442 20387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "375e8dc5e78547c18dd16755daecd7bb"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "dc935ddaeff3092300eb6781d06ff905"
server_key_iv: "f5e11941a9133ffec44514b4cf83c3d4"
server_key_version: "encryptionkey@0"
I20260808 18:21:01.499970 20387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:01.519769 20387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:01.522758 20387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:01.522943 20387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:01.529404 20387 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:33333
I20260808 18:21:01.529654 20506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:33333 every 8 connection(s)
I20260808 18:21:01.530407 20387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:01.534492 20509 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:21:01.536052 20509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap starting.
I20260808 18:21:01.538079 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 20387
I20260808 18:21:01.538582 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:43263
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:33333
--webserver_port=38653
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:21:01.553259 20509 log.cc:824] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:01.561244 20509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:21:01.561623 20509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap complete.
I20260808 18:21:01.565467 20509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:01.565807 20509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375e8dc5e78547c18dd16755daecd7bb, State: Initialized, Role: FOLLOWER
I20260808 18:21:01.566293 20509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:01.566404 20509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:01.566484 20509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:01.566601 20509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 1 FOLLOWER]: Advancing to term 2
I20260808 18:21:01.568390 20509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:01.568720 20509 leader_election.cc:302] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375e8dc5e78547c18dd16755daecd7bb; no voters:
I20260808 18:21:01.569012 20509 leader_election.cc:288] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 18:21:01.569128 20515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Leader election won for term 2
I20260808 18:21:01.569372 20515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 LEADER]: Becoming Leader. State: Replica: 375e8dc5e78547c18dd16755daecd7bb, State: Running, Role: LEADER
I20260808 18:21:01.569715 20515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:01.570190 20509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:01.571110 20516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "375e8dc5e78547c18dd16755daecd7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } } }
I20260808 18:21:01.571244 20516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:01.572185 20518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375e8dc5e78547c18dd16755daecd7bb. Latest consensus state: current_term: 2 leader_uuid: "375e8dc5e78547c18dd16755daecd7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } } }
I20260808 18:21:01.572366 20518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:01.574636 20543 catalog_manager.cc:1259] Loaded cluster ID: 6c7dcd4a2f374a43a1a78a0b168b5562
I20260808 18:21:01.574710 20543 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: loading cluster ID for follower catalog manager: success
I20260808 18:21:01.576413 20543 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: acquiring CA information for follower catalog manager: success
I20260808 18:21:01.577474 20543 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:21:01.577615 20531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:01.579830 20531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:01.580320 20531 catalog_manager.cc:1259] Loaded cluster ID: 6c7dcd4a2f374a43a1a78a0b168b5562
I20260808 18:21:01.580376 20531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:01.581027 20531 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:01.581549 20531 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Loaded TSK: 0
I20260808 18:21:01.600775 20531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Generated new TSK 1
I20260808 18:21:01.600983 20531 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:21:01.653368 20512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:01.653590 20512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:01.653631 20512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:01.657552 20512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:01.657600 20512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:01.657693 20512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:01.662803 20512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:43263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=38653
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:01.663739 20512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:01.664682 20512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:01.674875 20554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:01.674881 20555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:01.675114 20512 server_base.cc:1034] running on GCE node
W20260808 18:21:01.675254 20557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:01.675635 20512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:01.676185 20512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:01.677392 20512 hybrid_clock.cc:648] HybridClock initialized: now 1786213261677363 us; error 44 us; skew 500 ppm
Aug 08 18:21:01 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213261, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:01.680162 20512 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:21:01.681056 20512 webserver.cc:460] Webserver started at http://127.27.18.193:38653/ using document root <none> and password file <none>
I20260808 18:21:01.681588 20512 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:01.681694 20512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:01.685407 20512 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 18:21:01.687345 20569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:01.688584 20512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:01.688678 20512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "27c759855f9344ea9a0ce570fef54661"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "77114a726bd3df0c913d33e6b444978d"
server_key_iv: "d6dbb016833e7266f1cd6bc21800f878"
server_key_version: "encryptionkey@0"
I20260808 18:21:01.689157 20512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:01.709782 20512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:01.712450 20512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:01.712641 20512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:01.713210 20512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:01.714252 20582 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:21:01.718739 20512 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:21:01.718796 20512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 18:21:01.718873 20512 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:21:01.720446 20512 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:21:01.720492 20512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 18:21:01.720749 20582 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Bootstrap starting.
I20260808 18:21:01.730903 20512 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:43263
I20260808 18:21:01.730968 20767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:43263 every 8 connection(s)
I20260808 18:21:01.731866 20512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:01.735671 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 20512
I20260808 18:21:01.736274 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:43419
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:33333
--webserver_port=40917
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:21:01.739355 20582 log.cc:824] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:21:01 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213261, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:01.747430 20582 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:21:01.747898 20582 tablet_bootstrap.cc:492] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Bootstrap complete.
I20260808 18:21:01.749279 20582 ts_tablet_manager.cc:1397] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Time spent bootstrapping tablet: real 0.029s user 0.018s sys 0.008s
I20260808 18:21:01.750682 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:01.733972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:44229 (local address 127.27.18.254:33333)
0808 18:21:01.734453 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:01.734475 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:01.734704 (+ 229us) server_negotiation.cc:407] Connection header received
0808 18:21:01.735827 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:01.735841 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:01.736128 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:01.736430 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:01.737439 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.739014 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.740418 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.741187 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.741408 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:01.744633 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:01.744640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:01.744675 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:01.744710 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:01.746835 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:01.747603 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:01.747609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:01.747616 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:01.747773 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:01.748232 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:01.748238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:01.748243 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:01.748698 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:01.748831 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:01.749553 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 18:21:01.749836 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":113,"threads_started":1}
I20260808 18:21:01.751538 20582 raft_consensus.cc:348] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:21:01.752040 20582 raft_consensus.cc:729] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c759855f9344ea9a0ce570fef54661, State: Initialized, Role: FOLLOWER
I20260808 18:21:01.752156 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:01.733864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:44229)
0808 18:21:01.734528 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:01.734577 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:01.735341 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:01.736477 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:01.736484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:01.736873 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:01.737270 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:01.737282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.739187 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.739191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:01.740277 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:01.740288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.741362 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.741366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:01.741548 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:01.742388 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:01.742407 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:01.744426 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:01.746971 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:01.746979 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:01.746995 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:01.747419 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:01.747895 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:01.747901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:01.747906 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:01.748103 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:01.748835 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:01.748844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:01.749152 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 18:21:01.750934 (+ 1782us) client_negotiation.cc:240] Negotiation successful
0808 18:21:01.751264 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":141,"threads_started":1}
I20260808 18:21:01.752552 20582 consensus_queue.cc:260] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } }
I20260808 18:21:01.753453 20582 ts_tablet_manager.cc:1428] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:21:01.754465 20768 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33333
I20260808 18:21:01.754716 20768 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:01.755187 20768 heartbeater.cc:511] Master 127.27.18.254:33333 requested a full tablet report, sending...
I20260808 18:21:01.757612 20436 ts_manager.cc:194] Registered new tserver with Master: 27c759855f9344ea9a0ce570fef54661 (127.27.18.193:43263)
I20260808 18:21:01.761396 20436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:44229
I20260808 18:21:01.763752 20768 heartbeater.cc:503] Master 127.27.18.254:33333 was elected leader, sending a full tablet report...
Aug 08 18:21:01 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213261, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.193@KRBTEST.COM
I20260808 18:21:01.770947 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:01.760636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43263 (local address 127.0.0.1:47380)
0808 18:21:01.761286 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:01.761327 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:01.761509 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:01.761948 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:01.761954 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:01.762359 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:01.762502 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:01.762511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.763939 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.763942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:01.764730 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:01.764740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.765384 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.765388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:01.765499 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:01.766027 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:01.766038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:01.767523 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:01.769217 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:01.769226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:01.769231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:01.769450 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:01.769746 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:01.769750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:01.769752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:01.769866 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:01.770320 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:01.770326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:01.770441 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 18:21:01.770576 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 18:21:01.770752 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":110,"threads_started":1}
I20260808 18:21:01.771056 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:01.760683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.27.18.193:43263)
0808 18:21:01.761049 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:01.761059 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:01.761423 (+ 364us) server_negotiation.cc:407] Connection header received
0808 18:21:01.761651 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:01.761661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:01.761806 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:01.761946 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:01.762635 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.763822 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.764887 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.765265 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.765379 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:01.767652 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:01.767664 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:01.767678 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:01.767706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:01.769102 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:01.769563 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:01.769569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:01.769571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:01.769651 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:01.769948 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:01.769951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:01.769952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:01.770163 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:01.770333 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:01.770655 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 18:21:01.770804 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":282,"thread_start_us":89,"threads_started":1}
I20260808 18:21:01.771963 20651 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db99b6b20674737adfdbf0ff6569abb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:20:41 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:47380
I20260808 18:21:01.772864 20781 tablet_replica.cc:330] stopping tablet replica
I20260808 18:21:01.773138 20781 raft_consensus.cc:2229] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:01.773396 20781 raft_consensus.cc:2258] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:01.774351 20781 ts_tablet_manager.cc:1905] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:01.777624 20781 ts_tablet_manager.cc:1918] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:01.777714 20781 log.cc:1196] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal/wals/1db99b6b20674737adfdbf0ff6569abb
I20260808 18:21:01.777961 20781 ts_tablet_manager.cc:1939] T 1db99b6b20674737adfdbf0ff6569abb P 27c759855f9344ea9a0ce570fef54661: Deleting consensus metadata
I20260808 18:21:01.779489 20414 catalog_manager.cc:4841] TS 27c759855f9344ea9a0ce570fef54661 (127.27.18.193:43263): tablet 1db99b6b20674737adfdbf0ff6569abb (table test-table [id=1c3d1c635d194f05b1edde214abf5ff2]) successfully deleted
W20260808 18:21:01.852785 20772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:01.853000 20772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:01.853039 20772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:01.856921 20772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:01.856971 20772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:01.857023 20772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:01.861814 20772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:43419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=40917
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:01.862648 20772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:01.863591 20772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:01.872453 20791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:01.872488 20795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:01.872453 20790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:01.872792 20772 server_base.cc:1034] running on GCE node
I20260808 18:21:01.873186 20772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:01.873842 20772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:01.874998 20772 hybrid_clock.cc:648] HybridClock initialized: now 1786213261874985 us; error 23 us; skew 500 ppm
Aug 08 18:21:01 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213261, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:01.877688 20772 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:21:01.878499 20772 webserver.cc:460] Webserver started at http://127.27.18.194:40917/ using document root <none> and password file <none>
I20260808 18:21:01.878984 20772 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:01.879082 20772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:01.882645 20772 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:01.884713 20804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:01.885910 20772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:21:01.886035 20772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "8e1dd3d029d8413591c4f504eb7f4167"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "8eaa9274dea41897512aa59bf423cf0b"
server_key_iv: "544f6067a998de8a678c38098d879813"
server_key_version: "encryptionkey@0"
I20260808 18:21:01.886523 20772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:01.896992 20772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:01.900002 20772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:01.900219 20772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:01.900768 20772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:01.901767 20815 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 18:21:01.905615 20815 ts_tablet_manager.cc:1852] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 18:21:01.905978 20815 ts_tablet_manager.cc:1905] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:01.909554 20815 ts_tablet_manager.cc:1918] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:01.909650 20815 log.cc:1196] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/1db99b6b20674737adfdbf0ff6569abb
I20260808 18:21:01.910027 20815 ts_tablet_manager.cc:1939] T 1db99b6b20674737adfdbf0ff6569abb P 8e1dd3d029d8413591c4f504eb7f4167: Deleting consensus metadata
I20260808 18:21:01.910347 20772 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 18:21:01.910422 20772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260808 18:21:01.910516 20772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:01.910568 20772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:01.919562 20772 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:43419
I20260808 18:21:01.919616 20985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:43419 every 8 connection(s)
I20260808 18:21:01.920361 20772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:01.924358 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 20772
I20260808 18:21:01.924862 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:39453
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:33333
--webserver_port=36275
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 18:21:01 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213261, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:01.934288 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:01.921986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:50507 (local address 127.27.18.254:33333)
0808 18:21:01.922156 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:01.922161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:01.922936 (+ 775us) server_negotiation.cc:407] Connection header received
0808 18:21:01.923728 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:01.923731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:01.923782 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:01.923853 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:01.925290 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.925810 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.927262 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.927714 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.927841 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:01.930605 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:01.930608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:01.930610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:01.930635 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:01.932094 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:01.932735 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:01.932738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:01.932739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:01.932811 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:01.933203 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:01.933205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:01.933206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:01.933364 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:01.933437 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:01.934052 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 18:21:01.934175 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 18:21:01.935401 20988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:01.922339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:50507)
0808 18:21:01.922789 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:01.922826 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:01.923554 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:01.923998 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:01.924003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:01.924498 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:01.925120 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:01.925142 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.925942 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.925949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:01.927131 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:01.927141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:01.927853 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:01.927856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:01.927989 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:01.928581 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:01.928596 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:01.930432 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:01.932212 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:01.932221 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:01.932235 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:01.932634 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:01.932908 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:01.932913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:01.932916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:01.933102 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:01.933474 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:01.933482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:01.933795 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 18:21:01.934021 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 18:21:01.934378 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":102,"threads_started":1}
I20260808 18:21:01.936640 20986 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33333
I20260808 18:21:01.936866 20986 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:01.937337 20986 heartbeater.cc:511] Master 127.27.18.254:33333 requested a full tablet report, sending...
I20260808 18:21:01.938450 20436 ts_manager.cc:194] Registered new tserver with Master: 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419)
I20260808 18:21:01.939143 20436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:50507
W20260808 18:21:02.033609 20992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:02.033821 20992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:02.033859 20992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:02.037222 20992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:02.037259 20992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:02.037304 20992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:02.041962 20992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:39453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=36275
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:02.042734 20992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:02.043612 20992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:02.052394 21005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:02.052410 21001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:02.052462 20992 server_base.cc:1034] running on GCE node
W20260808 18:21:02.052412 21002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:02.053071 20992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:02.053606 20992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:02.054772 20992 hybrid_clock.cc:648] HybridClock initialized: now 1786213262054756 us; error 27 us; skew 500 ppm
Aug 08 18:21:02 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213262, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:02.057421 20992 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:21:02.058223 20992 webserver.cc:460] Webserver started at http://127.27.18.195:36275/ using document root <none> and password file <none>
I20260808 18:21:02.058740 20992 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:02.058836 20992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:02.062341 20992 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:21:02.064276 21013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:02.065495 20992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:02.065641 20992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "aededf31d0b94833bf11a929c29dcccf"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "4d986d55128328aac963ab688c216bdc"
server_key_iv: "e143a7109d3fb8816a574f55a839137d"
server_key_version: "encryptionkey@0"
I20260808 18:21:02.066125 20992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:02.077160 20992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:02.080109 20992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:02.080317 20992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:02.080854 20992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:02.081656 20992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:02.081722 20992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:02.081789 20992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:02.081840 20992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:02.090876 20992 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:39453
I20260808 18:21:02.090898 21178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:39453 every 8 connection(s)
I20260808 18:21:02.091813 20992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:02.092590 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 20992
Aug 08 18:21:02 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213262, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:02.101480 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.093290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39172 (local address 127.27.18.254:33333)
0808 18:21:02.093441 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.093446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.093666 (+ 220us) server_negotiation.cc:407] Connection header received
0808 18:21:02.093823 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.093827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.093879 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.093962 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:02.094758 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.095268 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.096279 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.096507 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.096628 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.098093 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:02.098097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:02.098099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:02.098125 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:02.099668 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:02.100110 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:02.100113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:02.100115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:02.100209 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:02.100491 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:02.100495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:02.100497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:02.100706 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:02.100918 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.101163 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.101292 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 18:21:02.105197 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.093590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:34107 (local address 127.27.18.254:33333)
0808 18:21:02.093790 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.093794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.094542 (+ 748us) server_negotiation.cc:407] Connection header received
0808 18:21:02.095789 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.095792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.095847 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.095939 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:02.096968 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.097714 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.099009 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.099359 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.099494 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.101916 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:02.101919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:02.101922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:02.101944 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:02.103527 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:02.103996 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:02.103998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:02.103999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:02.104067 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:02.104317 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:02.104319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:02.104320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:02.104496 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:02.104575 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.105003 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.105092 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260808 18:21:02.105931 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.093966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:34107)
0808 18:21:02.094391 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:02.094438 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:02.095605 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:02.096085 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:02.096092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:02.096391 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:02.096834 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.096847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.097861 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.097864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.098879 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.098893 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.099517 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.099520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.099656 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.100160 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:02.100175 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:02.101729 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:02.103633 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:02.103640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:02.103651 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:02.103899 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:02.104132 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:02.104134 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:02.104135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:02.104243 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:02.104589 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:02.104593 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:02.104806 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 18:21:02.104947 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 18:21:02.105167 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":130,"threads_started":1}
I20260808 18:21:02.106950 21179 heartbeater.cc:347] Connected to a master server at 127.27.18.254:33333
I20260808 18:21:02.107177 21179 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:02.107648 21179 heartbeater.cc:511] Master 127.27.18.254:33333 requested a full tablet report, sending...
I20260808 18:21:02.108561 20436 ts_manager.cc:194] Registered new tserver with Master: aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
I20260808 18:21:02.109052 20436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:34107
I20260808 18:21:02.114702 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:02.118453 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.115240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.27.18.254:33333)
0808 18:21:02.115401 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.115406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.115571 (+ 165us) server_negotiation.cc:407] Connection header received
0808 18:21:02.115762 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.115764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.115842 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.115942 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:02.116174 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.116619 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.117425 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.117617 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.117717 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.117998 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:02.118096 (+ 98us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 18:21:02.118344 (+ 248us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39174: token expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 18:21:02.118532 21182 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39174: token expired
W20260808 18:21:02.118822 20332 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.18.254:33333)
I20260808 18:21:02.125022 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.119155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.27.18.254:33333)
0808 18:21:02.119258 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.119262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.119281 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:21:02.119341 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.119343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.119426 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.119499 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:02.119979 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.120440 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.121153 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.121348 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.121417 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.122532 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:02.122534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:02.122536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:02.122555 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:02.123800 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:02.124154 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:02.124156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:02.124157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:02.124222 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:02.124467 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:02.124469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:02.124470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:02.124605 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:02.124699 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.124860 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.124936 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:21:02.126957 20436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:02.129957 20436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:02.137219 20651 tablet_service.cc:1467] Processing CreateTablet for tablet 102a1d60c72245d69da5022b20edcb91 (DEFAULT_TABLE table=test-table [id=e8b197c93cb3465682cfe18eda2ecc12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:02.137821 20651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102a1d60c72245d69da5022b20edcb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:02.140615 20582 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Bootstrap starting.
I20260808 18:21:02.140971 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.135952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43419 (local address 127.0.0.1:35064)
0808 18:21:02.136080 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:02.136095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:02.136177 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:02.137014 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:02.137017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:02.137034 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:02.137122 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.137128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.138456 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.138459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.139565 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.139574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.140557 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.140560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.140659 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.140708 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 18:21:02.140791 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 18:21:02.140861 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260808 18:21:02.141270 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.136174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.27.18.194:43419)
0808 18:21:02.136537 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.136550 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.136572 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:21:02.136650 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.136655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.136858 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.137036 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:02.137210 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.138336 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.139703 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.140421 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.140528 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.140664 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.140852 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.140963 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":152,"threads_started":1}
I20260808 18:21:02.141546 20582 tablet_bootstrap.cc:654] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:02.141607 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.136280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:39453 (local address 127.0.0.1:55818)
0808 18:21:02.136719 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:02.136732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:02.136819 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:02.137309 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:02.137312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:02.137330 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:02.137407 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.137413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.138711 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.138714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.139908 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.139919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.141176 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.141180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.141279 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.141320 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:21:02.141402 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 18:21:02.141482 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":44,"thread_start_us":52,"threads_started":1}
I20260808 18:21:02.141986 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.136482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.27.18.195:39453)
0808 18:21:02.136904 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.136916 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.136936 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:21:02.137001 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.137006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.137175 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.137324 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:02.137551 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.138503 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.140043 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.140872 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.140973 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.141104 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.141771 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.141860 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":85,"threads_started":1}
I20260808 18:21:02.142707 20885 tablet_service.cc:1467] Processing CreateTablet for tablet 102a1d60c72245d69da5022b20edcb91 (DEFAULT_TABLE table=test-table [id=e8b197c93cb3465682cfe18eda2ecc12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:02.143038 20582 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: No bootstrap required, opened a new log
I20260808 18:21:02.143146 20582 ts_tablet_manager.cc:1397] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 18:21:02.143632 20885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102a1d60c72245d69da5022b20edcb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:02.143608 20582 raft_consensus.cc:348] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.143713 20582 raft_consensus.cc:374] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:02.143760 20582 raft_consensus.cc:729] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c759855f9344ea9a0ce570fef54661, State: Initialized, Role: FOLLOWER
I20260808 18:21:02.143890 20582 consensus_queue.cc:260] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.144148 20582 ts_tablet_manager.cc:1428] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 18:21:02.145315 21085 tablet_service.cc:1467] Processing CreateTablet for tablet 102a1d60c72245d69da5022b20edcb91 (DEFAULT_TABLE table=test-table [id=e8b197c93cb3465682cfe18eda2ecc12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:02.146314 21085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102a1d60c72245d69da5022b20edcb91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:02.148381 20815 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Bootstrap starting.
I20260808 18:21:02.150418 20815 tablet_bootstrap.cc:654] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:02.151258 20815 log.cc:824] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:02.153152 20815 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: No bootstrap required, opened a new log
I20260808 18:21:02.153321 20815 ts_tablet_manager.cc:1397] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:02.153851 21195 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Bootstrap starting.
I20260808 18:21:02.156111 21195 tablet_bootstrap.cc:654] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:02.156354 20815 raft_consensus.cc:348] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.156538 20815 raft_consensus.cc:374] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:02.156592 20815 raft_consensus.cc:729] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e1dd3d029d8413591c4f504eb7f4167, State: Initialized, Role: FOLLOWER
I20260808 18:21:02.156852 21195 log.cc:824] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:02.157080 20815 consensus_queue.cc:260] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.157699 20986 heartbeater.cc:503] Master 127.27.18.254:33333 was elected leader, sending a full tablet report...
I20260808 18:21:02.158017 20815 ts_tablet_manager.cc:1428] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:21:02.158608 21195 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: No bootstrap required, opened a new log
I20260808 18:21:02.158785 21195 ts_tablet_manager.cc:1397] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:02.161810 21195 raft_consensus.cc:348] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.162038 21195 raft_consensus.cc:374] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:02.162106 21195 raft_consensus.cc:729] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aededf31d0b94833bf11a929c29dcccf, State: Initialized, Role: FOLLOWER
I20260808 18:21:02.162458 21195 consensus_queue.cc:260] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.163031 21179 heartbeater.cc:503] Master 127.27.18.254:33333 was elected leader, sending a full tablet report...
I20260808 18:21:02.163295 21195 ts_tablet_manager.cc:1428] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:21:02.168406 20774 raft_consensus.cc:482] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:02.168533 20774 raft_consensus.cc:504] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.169380 20774 leader_election.cc:288] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
W20260808 18:21:02.171185 20987 tablet.cc:2369] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:02.173390 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.169660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:35727 (local address 127.27.18.194:43419)
0808 18:21:02.169759 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.169763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.169778 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:21:02.169965 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.169968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.170003 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.170098 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:02.170346 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.170883 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.172294 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.172871 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.172981 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.173024 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.173202 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.173283 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:21:02.173408 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.169534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43419 (local address 127.27.18.193:35727)
0808 18:21:02.169662 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:02.169677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:02.169792 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:02.170112 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:02.170115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:02.170133 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:02.170243 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.170249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.171028 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.171031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.172191 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.172201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.172980 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.172982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.173061 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.173094 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 18:21:02.173185 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:21:02.173282 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":65}
I20260808 18:21:02.173772 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.169681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:39453 (local address 127.27.18.193:42473)
0808 18:21:02.169973 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:02.169987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:02.170118 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:02.170359 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:02.170362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:02.170377 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:02.170516 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.170524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.171413 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.171416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.172518 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:02.172528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.173332 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.173336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:02.173428 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.173453 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:21:02.173529 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 18:21:02.173621 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":22,"thread_start_us":114,"threads_started":1}
I20260808 18:21:02.173995 20910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a1d60c72245d69da5022b20edcb91" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1dd3d029d8413591c4f504eb7f4167" is_pre_election: true
I20260808 18:21:02.174082 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.169765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:42473 (local address 127.27.18.195:39453)
0808 18:21:02.169960 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.169964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.170070 (+ 106us) server_negotiation.cc:407] Connection header received
0808 18:21:02.170209 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.170212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.170254 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.170346 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:02.170630 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.171252 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.172632 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.173198 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.173307 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.173349 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.173917 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.173980 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 18:21:02.174222 20910 raft_consensus.cc:2454] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 0.
I20260808 18:21:02.174521 21116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a1d60c72245d69da5022b20edcb91" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aededf31d0b94833bf11a929c29dcccf" is_pre_election: true
I20260808 18:21:02.174633 20571 leader_election.cc:302] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661, 8e1dd3d029d8413591c4f504eb7f4167; no voters:
I20260808 18:21:02.174759 21116 raft_consensus.cc:2454] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 0.
I20260808 18:21:02.174911 20774 raft_consensus.cc:2781] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:02.174993 20774 raft_consensus.cc:482] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:02.175048 20774 raft_consensus.cc:3037] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:02.176002 20774 raft_consensus.cc:504] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.176304 20774 leader_election.cc:288] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 election: Requested vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
I20260808 18:21:02.176630 21116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a1d60c72245d69da5022b20edcb91" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aededf31d0b94833bf11a929c29dcccf"
I20260808 18:21:02.176608 20910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102a1d60c72245d69da5022b20edcb91" candidate_uuid: "27c759855f9344ea9a0ce570fef54661" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1dd3d029d8413591c4f504eb7f4167"
I20260808 18:21:02.176733 20910 raft_consensus.cc:3037] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:02.176733 21116 raft_consensus.cc:3037] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:02.177860 20910 raft_consensus.cc:2454] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 1.
I20260808 18:21:02.177860 21116 raft_consensus.cc:2454] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c759855f9344ea9a0ce570fef54661 in term 1.
I20260808 18:21:02.178123 20571 leader_election.cc:302] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661, 8e1dd3d029d8413591c4f504eb7f4167; no voters:
I20260808 18:21:02.178277 20774 raft_consensus.cc:2781] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:02.178473 20774 raft_consensus.cc:686] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 LEADER]: Becoming Leader. State: Replica: 27c759855f9344ea9a0ce570fef54661, State: Running, Role: LEADER
I20260808 18:21:02.178701 20774 consensus_queue.cc:237] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:02.181399 20436 catalog_manager.cc:5495] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27c759855f9344ea9a0ce570fef54661 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "27c759855f9344ea9a0ce570fef54661" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } health_report { overall_health: HEALTHY } } }
I20260808 18:21:02.198103 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.194852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.27.18.193:43263)
0808 18:21:02.194958 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:02.194964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:02.195029 (+ 65us) server_negotiation.cc:407] Connection header received
0808 18:21:02.195194 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:02.195196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:02.195239 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:02.195336 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:02.195590 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.196037 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.196793 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:02.197004 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:02.197120 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:02.197403 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:02.197729 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:02.197866 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:02.197937 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 18:21:02.197984 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:21:02.201296 21116 raft_consensus.cc:1261] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Refusing update from remote peer 27c759855f9344ea9a0ce570fef54661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:02.201315 20910 raft_consensus.cc:1261] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Refusing update from remote peer 27c759855f9344ea9a0ce570fef54661: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:02.201885 21206 consensus_queue.cc:1035] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:02.202019 20774 consensus_queue.cc:1035] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:02.212198 21212 mvcc.cc:205] Tried to move back new op lower bound from 7316329521971609600 to 7316329521886060544. Current Snapshot: MvccSnapshot[applied={T|T < 7316329521971609600}]
I20260808 18:21:02.216689 20434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:02.216836 20434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:02.218992 20434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Sending DeleteTablet for 3 replicas of tablet 102a1d60c72245d69da5022b20edcb91
I20260808 18:21:02.219627 20885 tablet_service.cc:1514] Processing DeleteTablet for tablet 102a1d60c72245d69da5022b20edcb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:02 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:35064
I20260808 18:21:02.219790 21085 tablet_service.cc:1514] Processing DeleteTablet for tablet 102a1d60c72245d69da5022b20edcb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:02 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:55818
I20260808 18:21:02.220006 20651 tablet_service.cc:1514] Processing DeleteTablet for tablet 102a1d60c72245d69da5022b20edcb91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:02 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:47380
I20260808 18:21:02.220131 21220 tablet_replica.cc:330] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: stopping tablet replica
I20260808 18:21:02.220196 20781 tablet_replica.cc:330] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: stopping tablet replica
I20260808 18:21:02.220319 20781 raft_consensus.cc:2229] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:21:02.220515 21220 raft_consensus.cc:2229] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:02.220557 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 20512
I20260808 18:21:02.220562 20781 raft_consensus.cc:2258] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:02.224355 21220 raft_consensus.cc:2258] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:02.224658 21221 tablet_replica.cc:330] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: stopping tablet replica
I20260808 18:21:02.224957 21221 raft_consensus.cc:2229] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:02.225178 21221 raft_consensus.cc:2258] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:02.226161 21220 ts_tablet_manager.cc:1905] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:02.229094 21220 ts_tablet_manager.cc:1918] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:02.229158 21220 log.cc:1196] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/102a1d60c72245d69da5022b20edcb91
I20260808 18:21:02.229802 21220 ts_tablet_manager.cc:1939] T 102a1d60c72245d69da5022b20edcb91 P 8e1dd3d029d8413591c4f504eb7f4167: Deleting consensus metadata
I20260808 18:21:02.230768 21221 ts_tablet_manager.cc:1905] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:02.231099 20413 catalog_manager.cc:4841] TS 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419): tablet 102a1d60c72245d69da5022b20edcb91 (table test-table [id=e8b197c93cb3465682cfe18eda2ecc12]) successfully deleted
I20260808 18:21:02.233161 21221 ts_tablet_manager.cc:1918] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:02.233237 21221 log.cc:1196] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/102a1d60c72245d69da5022b20edcb91
I20260808 18:21:02.233551 21221 ts_tablet_manager.cc:1939] T 102a1d60c72245d69da5022b20edcb91 P aededf31d0b94833bf11a929c29dcccf: Deleting consensus metadata
I20260808 18:21:02.234489 20414 catalog_manager.cc:4841] TS aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453): tablet 102a1d60c72245d69da5022b20edcb91 (table test-table [id=e8b197c93cb3465682cfe18eda2ecc12]) successfully deleted
W20260808 18:21:02.238821 20414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:43263 (error 108)
I20260808 18:21:02.239157 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 20772
I20260808 18:21:02.243561 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:02.243138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:43263 (local address 127.0.0.1:47400)
0808 18:21:02.243324 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:02.243450 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:43263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 18:21:02.243935 20414 catalog_manager.cc:4584] TS 27c759855f9344ea9a0ce570fef54661 (127.27.18.193:43263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 102a1d60c72245d69da5022b20edcb91: Network error: Client connection negotiation failed: client connection to 127.27.18.193:43263: connect: Connection refused (error 111)
I20260808 18:21:02.251837 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 20992
I20260808 18:21:02.260053 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 20387
I20260808 18:21:22.269444 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33333
--webserver_interface=127.27.18.254
--webserver_port=45695
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:22.383543 21223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:22.383767 21223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:22.383808 21223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:22.387874 21223 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 18:21:22.387917 21223 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 18:21:22.387933 21223 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 18:21:22.387945 21223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:22.387995 21223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:22.388025 21223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:22.388055 21223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:22.388111 21223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:22.392964 21223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=45695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:22.393894 21223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:22.394855 21223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:22.403215 21236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:22.403415 21232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:22.403230 21234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:22.403414 21223 server_base.cc:1034] running on GCE node
I20260808 18:21:22.404098 21223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:22.405134 21223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:22.406333 21223 hybrid_clock.cc:648] HybridClock initialized: now 1786213282406326 us; error 38 us; skew 500 ppm
Aug 08 18:21:22 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213282, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:22.408975 21223 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:21:22.409929 21223 webserver.cc:460] Webserver started at http://127.27.18.254:45695/ using document root <none> and password file <none>
I20260808 18:21:22.410475 21223 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:22.410570 21223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:22.414201 21223 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 18:21:22.416290 21247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:22.417455 21223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:22.417608 21223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "375e8dc5e78547c18dd16755daecd7bb"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "dc935ddaeff3092300eb6781d06ff905"
server_key_iv: "f5e11941a9133ffec44514b4cf83c3d4"
server_key_version: "encryptionkey@0"
I20260808 18:21:22.418133 21223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:22.433367 21223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:22.436010 21223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:22.436192 21223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:22.443198 21223 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:33333
I20260808 18:21:22.443547 21326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:33333 every 8 connection(s)
I20260808 18:21:22.444334 21223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:22.448565 21327 sys_catalog.cc:263] Verifying existing consensus state
I20260808 18:21:22.448873 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 21223
I20260808 18:21:22.449421 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:43263
--local_ip_for_outbound_sockets=127.27.18.193
--tserver_master_addrs=127.27.18.254:33333
--webserver_port=38653
--webserver_interface=127.27.18.193
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:21:22.450865 21327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap starting.
I20260808 18:21:22.469033 21327 log.cc:824] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:22.479970 21327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:21:22.480329 21327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Bootstrap complete.
I20260808 18:21:22.484366 21327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:22.484848 21327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375e8dc5e78547c18dd16755daecd7bb, State: Initialized, Role: FOLLOWER
I20260808 18:21:22.485360 21327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:22.485565 21327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:22.485649 21327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:22.485767 21327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 2 FOLLOWER]: Advancing to term 3
I20260808 18:21:22.487839 21327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:22.488165 21327 leader_election.cc:302] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375e8dc5e78547c18dd16755daecd7bb; no voters:
I20260808 18:21:22.488468 21327 leader_election.cc:288] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 18:21:22.488595 21332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 3 FOLLOWER]: Leader election won for term 3
I20260808 18:21:22.488819 21332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [term 3 LEADER]: Becoming Leader. State: Replica: 375e8dc5e78547c18dd16755daecd7bb, State: Running, Role: LEADER
I20260808 18:21:22.489176 21332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } }
I20260808 18:21:22.489665 21327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:22.490115 21332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375e8dc5e78547c18dd16755daecd7bb. Latest consensus state: current_term: 3 leader_uuid: "375e8dc5e78547c18dd16755daecd7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } } }
I20260808 18:21:22.490851 21332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:22.490742 21334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "375e8dc5e78547c18dd16755daecd7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375e8dc5e78547c18dd16755daecd7bb" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33333 } } }
I20260808 18:21:22.491508 21334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:22.491945 21352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:22.496734 21352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:22.497347 21352 catalog_manager.cc:1259] Loaded cluster ID: 6c7dcd4a2f374a43a1a78a0b168b5562
I20260808 18:21:22.497421 21352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:22.499864 21352 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:22.500583 21352 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Loaded TSK: 0
I20260808 18:21:22.500660 21352 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 375e8dc5e78547c18dd16755daecd7bb: Loaded TSK: 1
F20260808 18:21:22.501395 21352 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6a06ddb9ad google::LogMessage::Fail() at ??:0
@ 0x7f6a06dde88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6a06ddb4de google::LogMessage::Flush() at ??:0
@ 0x7f6a06ddca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6a09a9f18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6a09a97f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6a117407bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6a1171a99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6a1171b178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f6a1175cdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6a1177b82b std::function<>::operator()() at ??:0
@ 0x7f6a1171ab4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6a1171bf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6a11718579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6a1175c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6a0902d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6a0902dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6a0902f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6a0901e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6a0a6626db start_thread at ??:0
@ 0x7f6a058d961f clone at ??:0
W20260808 18:21:22.599007 21330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:22.599344 21330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:22.599449 21330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:22.603230 21330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:22.603317 21330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:22.603441 21330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:22.608567 21330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:43263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=38653
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:22.609575 21330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:22.610572 21330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:22.620023 21404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:22.620033 21403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:22.620162 21406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:22.620471 21330 server_base.cc:1034] running on GCE node
I20260808 18:21:22.621127 21330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:22.621786 21330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:22.622982 21330 hybrid_clock.cc:648] HybridClock initialized: now 1786213282622954 us; error 45 us; skew 500 ppm
Aug 08 18:21:22 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213282, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:22.629007 21330 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:21:22.629885 21330 webserver.cc:460] Webserver started at http://127.27.18.193:38653/ using document root <none> and password file <none>
I20260808 18:21:22.630460 21330 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:22.630522 21330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:22.634433 21330 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:22.636572 21425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:22.637840 21330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:21:22.637972 21330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "27c759855f9344ea9a0ce570fef54661"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "77114a726bd3df0c913d33e6b444978d"
server_key_iv: "d6dbb016833e7266f1cd6bc21800f878"
server_key_version: "encryptionkey@0"
I20260808 18:21:22.638492 21330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:22.653561 21330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:22.656674 21330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:22.656888 21330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:22.657569 21330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:22.658802 21434 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 18:21:22.664248 21330 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 18:21:22.664324 21330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 18:21:22.664433 21330 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 18:21:22.666029 21330 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 18:21:22.666100 21330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 18:21:22.666388 21434 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Bootstrap starting.
I20260808 18:21:22.677304 21330 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:43263
I20260808 18:21:22.678162 21330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:22.679026 21595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:43263 every 8 connection(s)
I20260808 18:21:22.687568 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 21330
I20260808 18:21:22.688006 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:43419
--local_ip_for_outbound_sockets=127.27.18.194
--tserver_master_addrs=127.27.18.254:33333
--webserver_port=40917
--webserver_interface=127.27.18.194
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:21:22.689138 21434 log.cc:824] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:22.697853 21434 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 18:21:22.698640 21434 tablet_bootstrap.cc:492] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Bootstrap complete.
I20260808 18:21:22.700282 21434 ts_tablet_manager.cc:1397] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.007s
I20260808 18:21:22.703652 21434 raft_consensus.cc:348] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:22.704159 21434 raft_consensus.cc:729] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c759855f9344ea9a0ce570fef54661, State: Initialized, Role: FOLLOWER
I20260808 18:21:22.704681 21434 consensus_queue.cc:260] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:22.705659 21434 ts_tablet_manager.cc:1428] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 18:21:22.733307 21600 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:21:22.680197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:60625)
0808 18:21:22.680687 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:22.680736 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:22.681496 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:22.732567 (+ 51071us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":160,"threads_started":1}
W20260808 18:21:22.733904 21598 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:33333 (0 consecutive failures): Network error: Failed to ping master at 127.27.18.254:33333: Client connection negotiation failed: client connection to 127.27.18.254:33333: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 18:21:22.734638 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:22.734378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:33007)
0808 18:21:22.734512 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:22.734585 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 18:21:22.735325 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:22.735113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:37515)
0808 18:21:22.735221 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:22.735266 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:21:22.735553 21598 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:21:22.806932 21604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:22.807153 21604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:22.807192 21604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:22.810956 21604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:22.811003 21604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:22.811055 21604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:22.815817 21604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:43419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=40917
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:22.816880 21604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:22.817757 21604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:22.827131 21615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:22.827220 21614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:22.827129 21619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:22.827391 21604 server_base.cc:1034] running on GCE node
I20260808 18:21:22.827800 21604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:22.828409 21604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:22.829572 21604 hybrid_clock.cc:648] HybridClock initialized: now 1786213282829569 us; error 37 us; skew 500 ppm
Aug 08 18:21:22 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213282, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:22.832232 21604 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:21:22.832978 21604 webserver.cc:460] Webserver started at http://127.27.18.194:40917/ using document root <none> and password file <none>
I20260808 18:21:22.833499 21604 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:22.833555 21604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:22.837075 21604 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:21:22.838883 21629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:22.840095 21604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:22.840188 21604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "8e1dd3d029d8413591c4f504eb7f4167"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "8eaa9274dea41897512aa59bf423cf0b"
server_key_iv: "544f6067a998de8a678c38098d879813"
server_key_version: "encryptionkey@0"
I20260808 18:21:22.840663 21604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:22.851362 21604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:22.854624 21604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:22.854774 21604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:22.855273 21604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:22.856079 21604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:22.856125 21604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:22.856170 21604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:22.856191 21604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:22.864790 21604 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:43419
I20260808 18:21:22.864818 21813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:43419 every 8 connection(s)
I20260808 18:21:22.865743 21604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:22.867660 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 21604
I20260808 18:21:22.868085 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:39453
--local_ip_for_outbound_sockets=127.27.18.195
--tserver_master_addrs=127.27.18.254:33333
--webserver_port=36275
--webserver_interface=127.27.18.195
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:21:22.869534 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:22.867909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:42507)
0808 18:21:22.868383 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:22.868905 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
W20260808 18:21:22.870127 21814 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:33333 (0 consecutive failures): Network error: Failed to ping master at 127.27.18.254:33333: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
I20260808 18:21:22.870832 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:22.870576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:57781)
0808 18:21:22.870701 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:22.870762 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 18:21:22.871516 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:22.871258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:55459)
0808 18:21:22.871364 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:22.871446 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:21:22.871690 21814 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 18:21:22.979205 21819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:22.979542 21819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:22.979630 21819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:22.984423 21819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:22.984506 21819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:22.984596 21819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:22.989818 21819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:39453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=36275
--tserver_master_addrs=127.27.18.254:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:22.990820 21819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:22.991904 21819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:23.001358 21828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:23.001513 21819 server_base.cc:1034] running on GCE node
W20260808 18:21:23.001382 21826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:23.001396 21830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:23.002166 21819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:23.002822 21819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:23.003983 21819 hybrid_clock.cc:648] HybridClock initialized: now 1786213283003969 us; error 24 us; skew 500 ppm
Aug 08 18:21:23 dist-test-slave-x4qh krb5kdc[19511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213283, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:23.006778 21819 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:21:23.007761 21819 webserver.cc:460] Webserver started at http://127.27.18.195:36275/ using document root <none> and password file <none>
I20260808 18:21:23.008322 21819 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:23.008423 21819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:23.012121 21819 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 18:21:23.014125 21837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:23.015267 21819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:23.015511 21819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "aededf31d0b94833bf11a929c29dcccf"
format_stamp: "Formatted at 2026-08-08 18:20:40 on dist-test-slave-x4qh"
server_key: "4d986d55128328aac963ab688c216bdc"
server_key_iv: "e143a7109d3fb8816a574f55a839137d"
server_key_version: "encryptionkey@0"
I20260808 18:21:23.016057 21819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:23.039811 21819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:23.042906 21819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:23.043104 21819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:23.043697 21819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:23.044603 21819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:23.044677 21819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:23.044740 21819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:23.044773 21819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:23.053670 21819 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:39453
I20260808 18:21:23.053716 22004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:39453 every 8 connection(s)
I20260808 18:21:23.054610 21819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:23.055704 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 21819
I20260808 18:21:23.058240 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.056719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:46469)
0808 18:21:23.057214 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.057611 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":123,"threads_started":1}
W20260808 18:21:23.058660 22005 heartbeater.cc:644] Failed to heartbeat to 127.27.18.254:33333 (0 consecutive failures): Network error: Failed to ping master at 127.27.18.254:33333: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
I20260808 18:21:23.059159 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.058971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:41487)
0808 18:21:23.059068 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.059111 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 18:21:23.059737 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.059575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:57711)
0808 18:21:23.059660 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.059697 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 18:21:23.059897 22005 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 18:21:23.736495 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.736069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:38675)
0808 18:21:23.736344 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.736395 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":72,"threads_started":1}
I20260808 18:21:23.872530 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.872159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:35833)
0808 18:21:23.872391 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.872440 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":66,"threads_started":1}
I20260808 18:21:23.933243 22013 raft_consensus.cc:482] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:23.933454 22013 raft_consensus.cc:504] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:23.934505 22013 leader_election.cc:288] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
Aug 08 18:21:23 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213282, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.195@KRBTEST.COM
Aug 08 18:21:23 dist-test-slave-x4qh krb5kdc[19511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213282, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.194@KRBTEST.COM
I20260808 18:21:23.948130 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.934726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:39453 (local address 127.27.18.193:55735)
0808 18:21:23.935363 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.935430 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:23.935552 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:23.936760 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:23.936763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:23.937285 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:23.937708 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:23.937721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.939047 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.939050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:23.939963 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:23.939973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.940654 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.940658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:23.940909 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:23.941657 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:23.941673 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:23.943603 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:23.945720 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:23.945725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:23.945737 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:23.945986 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:23.946390 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:23.946396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:23.946537 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:23.947147 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:23.947153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:23.947450 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 18:21:23.947634 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 18:21:23.947948 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":284,"spinlock_wait_cycles":4224,"thread_start_us":60,"threads_started":1}
I20260808 18:21:23.948191 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.934925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:55735 (local address 127.27.18.195:39453)
0808 18:21:23.935266 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:23.935284 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:23.935528 (+ 244us) server_negotiation.cc:407] Connection header received
0808 18:21:23.936207 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:23.936212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:23.936541 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:23.936815 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:23.937877 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.938872 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.940087 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.940521 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.940692 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:23.943829 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:23.943834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:23.943844 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:23.943870 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:23.945595 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946158 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:23.946162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:23.946166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:23.946276 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946643 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:23.946646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:23.946648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:23.947007 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:23.947173 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:23.947764 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 18:21:23.948037 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":121,"threads_started":1}
I20260808 18:21:23.948232 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.934945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:58985 (local address 127.27.18.194:43419)
0808 18:21:23.935279 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:23.935299 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:23.935335 (+ 36us) server_negotiation.cc:407] Connection header received
0808 18:21:23.936051 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:23.936056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:23.936394 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:23.936705 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:23.937886 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.938872 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.940085 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.940522 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.940693 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:23.944057 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:23.944061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:23.944094 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:23.944130 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:23.945991 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946477 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:23.946483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:23.946488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:23.946600 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946895 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:23.946898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:23.946900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:23.947259 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:23.947419 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:23.947771 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 18:21:23.948052 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":102,"threads_started":1}
I20260808 18:21:23.949368 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:23.934726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43419 (local address 127.27.18.193:58985)
0808 18:21:23.934869 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:23.934884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:23.934992 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:23.936753 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:23.936759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:23.937274 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:23.937708 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:23.937721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.939015 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.939018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:23.939961 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:23.939970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:23.940650 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:23.940654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:23.940910 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:23.941616 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:23.941658 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:23.943883 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:23.946143 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:23.946148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:23.946359 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:23.946699 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:23.946703 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:23.946705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:23.946791 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:23.947395 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:23.947398 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:23.947508 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 18:21:23.947636 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 18:21:23.948047 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":261,"spinlock_wait_cycles":29184}
W20260808 18:21:23.950845 21428 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
W20260808 18:21:23.951227 21427 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
I20260808 18:21:23.951347 21427 leader_election.cc:302] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661; no voters: 8e1dd3d029d8413591c4f504eb7f4167, aededf31d0b94833bf11a929c29dcccf
I20260808 18:21:23.951610 22013 raft_consensus.cc:2726] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:21:24.060861 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:24.060373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:49595)
0808 18:21:24.060662 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:24.060725 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
I20260808 18:21:24.737757 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:24.737359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:43167)
0808 18:21:24.737609 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:24.737661 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260808 18:21:24.873553 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:24.873193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:41965)
0808 18:21:24.873431 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:24.873479 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260808 18:21:25.062112 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:25.061714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:58735)
0808 18:21:25.061967 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:25.062017 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260808 18:21:25.512799 22027 raft_consensus.cc:482] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:25.513000 22027 raft_consensus.cc:504] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:25.513578 22027 leader_election.cc:288] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
W20260808 18:21:25.514621 21428 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
W20260808 18:21:25.514744 21427 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
I20260808 18:21:25.514840 21427 leader_election.cc:302] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661; no voters: 8e1dd3d029d8413591c4f504eb7f4167, aededf31d0b94833bf11a929c29dcccf
I20260808 18:21:25.514973 22027 raft_consensus.cc:2726] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:21:25.739154 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:25.738643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:36315)
0808 18:21:25.738986 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:25.739052 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":86,"threads_started":1}
I20260808 18:21:25.874909 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:25.874410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:38971)
0808 18:21:25.874751 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:25.874812 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":72,"threads_started":1}
I20260808 18:21:26.063354 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:26.062909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:39337)
0808 18:21:26.063210 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:26.063270 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":72,"threads_started":1}
I20260808 18:21:26.740257 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:26.739889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:46873)
0808 18:21:26.740115 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:26.740164 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
I20260808 18:21:26.876123 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:26.875751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:50363)
0808 18:21:26.875998 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:26.876046 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260808 18:21:27.064509 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:27.064102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:37059)
0808 18:21:27.064358 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:27.064427 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260808 18:21:27.741593 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:27.741128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:60975)
0808 18:21:27.741454 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:27.741503 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260808 18:21:27.877197 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:27.876831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:37679)
0808 18:21:27.877063 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:27.877113 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260808 18:21:28.065631 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:28.065230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:56665)
0808 18:21:28.065471 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:28.065534 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260808 18:21:28.742664 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:28.742249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:42903)
0808 18:21:28.742521 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:28.742572 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
I20260808 18:21:28.761250 22042 raft_consensus.cc:482] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:28.761370 22042 raft_consensus.cc:504] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:28.761823 22042 leader_election.cc:288] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
W20260808 18:21:28.762625 21428 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
W20260808 18:21:28.762720 21427 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
I20260808 18:21:28.762817 21427 leader_election.cc:302] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661; no voters: 8e1dd3d029d8413591c4f504eb7f4167, aededf31d0b94833bf11a929c29dcccf
I20260808 18:21:28.762984 22042 raft_consensus.cc:2726] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:21:28.878394 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:28.877997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:34227)
0808 18:21:28.878242 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:28.878310 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260808 18:21:29.066682 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:29.066298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:59691)
0808 18:21:29.066549 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:29.066600 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260808 18:21:29.743904 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:29.743442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:46861)
0808 18:21:29.743749 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:29.743805 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260808 18:21:29.879565 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:29.879069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:51809)
0808 18:21:29.879423 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:29.879480 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":68,"threads_started":1}
I20260808 18:21:30.067865 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:30.067444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:52749)
0808 18:21:30.067711 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:30.067776 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":67,"threads_started":1}
I20260808 18:21:30.744943 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:30.744558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:38855)
0808 18:21:30.744795 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:30.744847 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 18:21:30.880574 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:30.880205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:39533)
0808 18:21:30.880438 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:30.880488 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260808 18:21:31.068923 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:31.068531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:34013)
0808 18:21:31.068777 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:31.068844 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":68,"threads_started":1}
I20260808 18:21:31.745973 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:31.745579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:42761)
0808 18:21:31.745835 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:31.745883 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
I20260808 18:21:31.881898 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:31.881506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:57367)
0808 18:21:31.881767 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:31.881819 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260808 18:21:32.070215 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:32.069831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:41119)
0808 18:21:32.070087 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:32.070137 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":97,"threads_started":1}
I20260808 18:21:32.747193 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:32.746788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:56241)
0808 18:21:32.747042 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:32.747095 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260808 18:21:32.883006 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:32.882592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:50313)
0808 18:21:32.882861 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:32.882913 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":71,"threads_started":1}
I20260808 18:21:33.071547 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:33.071071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:58609)
0808 18:21:33.071307 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:33.071369 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":73,"threads_started":1}
I20260808 18:21:33.202200 22065 raft_consensus.cc:482] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:33.202346 22065 raft_consensus.cc:504] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1dd3d029d8413591c4f504eb7f4167" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43419 } } peers { permanent_uuid: "aededf31d0b94833bf11a929c29dcccf" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39453 } } peers { permanent_uuid: "27c759855f9344ea9a0ce570fef54661" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 43263 } }
I20260808 18:21:33.202831 22065 leader_election.cc:288] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419), aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453)
W20260808 18:21:33.203624 21428 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer aededf31d0b94833bf11a929c29dcccf (127.27.18.195:39453): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
W20260808 18:21:33.203723 21427 leader_election.cc:341] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e1dd3d029d8413591c4f504eb7f4167 (127.27.18.194:43419): Not found: Tablet not found: 102a1d60c72245d69da5022b20edcb91
I20260808 18:21:33.203820 21427 leader_election.cc:302] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27c759855f9344ea9a0ce570fef54661; no voters: 8e1dd3d029d8413591c4f504eb7f4167, aededf31d0b94833bf11a929c29dcccf
I20260808 18:21:33.203951 22065 raft_consensus.cc:2726] T 102a1d60c72245d69da5022b20edcb91 P 27c759855f9344ea9a0ce570fef54661 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 18:21:33.748293 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:33.747916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:35071)
0808 18:21:33.748149 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:33.748200 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260808 18:21:33.884135 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:33.883718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:54013)
0808 18:21:33.883985 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:33.884036 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260808 18:21:34.072574 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:34.072224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:38361)
0808 18:21:34.072449 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:34.072497 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260808 18:21:34.749522 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:34.749031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:40609)
0808 18:21:34.749346 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:34.749408 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":98,"threads_started":1}
I20260808 18:21:34.885366 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:34.884912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:52031)
0808 18:21:34.885170 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:34.885246 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
I20260808 18:21:35.073727 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:35.073313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:52543)
0808 18:21:35.073587 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:35.073639 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
I20260808 18:21:35.750772 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:35.750361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:35427)
0808 18:21:35.750609 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:35.750663 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260808 18:21:35.886514 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:35.886139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:34155)
0808 18:21:35.886379 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:35.886429 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":65,"threads_started":1}
I20260808 18:21:36.074843 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:36.074464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:40643)
0808 18:21:36.074702 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:36.074749 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":85,"threads_started":1}
I20260808 18:21:36.752215 22077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:36.751780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:33499)
0808 18:21:36.752066 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:36.752120 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":89,"threads_started":1}
I20260808 18:21:36.887549 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:36.887149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:35943)
0808 18:21:36.887402 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:36.887450 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":84,"threads_started":1}
I20260808 18:21:37.076052 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:37.075627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:33953)
0808 18:21:37.075891 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:37.075953 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260808 18:21:37.753607 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:37.753108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.193:49941)
0808 18:21:37.753467 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:37.753513 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":103,"threads_started":1}
I20260808 18:21:37.888720 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:37.888344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.194:42131)
0808 18:21:37.888587 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:37.888635 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260808 18:21:38.063295 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 21330
I20260808 18:21:38.075012 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 21604
I20260808 18:21:38.080483 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:38.078331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:33333 (local address 127.27.18.195:36589)
0808 18:21:38.080268 (+ 1937us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:38.080337 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.254:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1838,"thread_start_us":721,"threads_started":1}
I20260808 18:21:38.086113 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 21819
2026-08-08T18:21:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59354 ms)
[----------] 30 tests from SecurityITest (239649 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:38 dist-test-slave-x4qh krb5kdc[22086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:38 dist-test-slave-x4qh krb5kdc[22086](info): set up 2 sockets
Aug 08 18:21:38 dist-test-slave-x4qh krb5kdc[22086](info): commencing operation
krb5kdc: starting...
W20260808 18:21:39.252701 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:39 dist-test-slave-x4qh krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:21:39.368587 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35667
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:35667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:39.474195 22105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:39.474467 22105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:39.474555 22105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:39.478266 22105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:39.478344 22105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:39.478382 22105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:39.478440 22105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:39.478478 22105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:39.483302 22105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:35667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:39.484315 22105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:39.485332 22105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:21:39.493778 22105 server_base.cc:1034] running on GCE node
W20260808 18:21:39.493773 22121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:39.493772 22117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:39.493794 22116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:39.494529 22105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:39.495639 22105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:39.496838 22105 hybrid_clock.cc:648] HybridClock initialized: now 1786213299496820 us; error 29 us; skew 500 ppm
Aug 08 18:21:39 dist-test-slave-x4qh krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:39.499501 22105 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:21:39.500334 22105 webserver.cc:460] Webserver started at http://127.27.18.254:44521/ using document root <none> and password file <none>
I20260808 18:21:39.500880 22105 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:39.500970 22105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:39.501197 22105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:39.503086 22105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "1a28de6412494ba3b35fd64e55a1bc04"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "81bcdcc8a8c58e53de47cf0d642841e8"
server_key_iv: "7d82c16727a49cab46d622451146d9bf"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.503693 22105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "1a28de6412494ba3b35fd64e55a1bc04"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "81bcdcc8a8c58e53de47cf0d642841e8"
server_key_iv: "7d82c16727a49cab46d622451146d9bf"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.507915 22105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 18:21:39.511124 22129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:39.512647 22105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:21:39.512777 22105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "1a28de6412494ba3b35fd64e55a1bc04"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "81bcdcc8a8c58e53de47cf0d642841e8"
server_key_iv: "7d82c16727a49cab46d622451146d9bf"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.512894 22105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:39.533772 22105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:39.536377 22105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:39.536558 22105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:39.543521 22105 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:35667
I20260808 18:21:39.543573 22200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:35667 every 8 connection(s)
I20260808 18:21:39.544435 22105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:39.546893 22201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:39.552141 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: Bootstrap starting.
I20260808 18:21:39.554477 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 22105
I20260808 18:21:39.554540 22201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:39.554617 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:39.554958 27723 external_mini_cluster.cc:1428] Setting key ab96f6e282efa479f46de5274e026bc2
I20260808 18:21:39.555435 22201 log.cc:824] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:39.557399 22201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: No bootstrap required, opened a new log
I20260808 18:21:39.560688 22201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28de6412494ba3b35fd64e55a1bc04" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35667 } }
I20260808 18:21:39.560885 22201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:39.560940 22201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a28de6412494ba3b35fd64e55a1bc04, State: Initialized, Role: FOLLOWER
I20260808 18:21:39.561424 22201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28de6412494ba3b35fd64e55a1bc04" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35667 } }
I20260808 18:21:39.561569 22201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:39.561685 22201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:39.561800 22201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:39.562994 22201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28de6412494ba3b35fd64e55a1bc04" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35667 } }
I20260808 18:21:39.563309 22201 leader_election.cc:302] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a28de6412494ba3b35fd64e55a1bc04; no voters:
Aug 08 18:21:39 dist-test-slave-x4qh krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:39.563688 22201 leader_election.cc:288] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:39.563793 22210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:39.564051 22210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [term 1 LEADER]: Becoming Leader. State: Replica: 1a28de6412494ba3b35fd64e55a1bc04, State: Running, Role: LEADER
I20260808 18:21:39.564425 22210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28de6412494ba3b35fd64e55a1bc04" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35667 } }
I20260808 18:21:39.564955 22201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:39.566784 22212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a28de6412494ba3b35fd64e55a1bc04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28de6412494ba3b35fd64e55a1bc04" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35667 } } }
I20260808 18:21:39.566913 22212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:39.567239 22212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a28de6412494ba3b35fd64e55a1bc04. Latest consensus state: current_term: 1 leader_uuid: "1a28de6412494ba3b35fd64e55a1bc04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28de6412494ba3b35fd64e55a1bc04" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35667 } } }
I20260808 18:21:39.567469 22212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:39.569918 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:39.556111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.27.18.254:35667)
0808 18:21:39.556491 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:39.556504 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:39.556536 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:21:39.557093 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:39.557107 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:39.557372 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:39.557720 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:39.558474 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:39.559550 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:39.560396 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:39.560861 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:39.561046 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:39.564297 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:39.564303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:39.564317 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:39.564360 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:39.566573 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:39.567340 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:39.567347 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:39.567355 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:39.567523 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:39.567871 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:39.567878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:39.567883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:39.568368 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:39.568530 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:39.568808 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 18:21:39.569056 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
W20260808 18:21:39.570309 22229 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:21:39.570390 22229 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:21:39.570484 22230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:39.571432 22230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:39.576737 22230 catalog_manager.cc:1347] Generated new cluster ID: ca976742fea047e8bf6ee6b03c017e4c
I20260808 18:21:39.576795 22230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:39.586378 22230 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:39.587481 22230 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:39.594231 22230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: Generated new TSK 0
I20260808 18:21:39.594810 22230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:21:39.623095 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35667
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:39.730011 22240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:39.730224 22240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:39.730262 22240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:39.734022 22240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:39.734071 22240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:39.734164 22240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:39.739943 22240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:39.740799 22240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:39.741725 22240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:39.750780 22252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:39.750753 22247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:39.750979 22240 server_base.cc:1034] running on GCE node
W20260808 18:21:39.750770 22246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:39.751502 22240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:39.752038 22240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:39.753211 22240 hybrid_clock.cc:648] HybridClock initialized: now 1786213299753181 us; error 41 us; skew 500 ppm
Aug 08 18:21:39 dist-test-slave-x4qh krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:39.755789 22240 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:21:39.756554 22240 webserver.cc:460] Webserver started at http://127.27.18.193:41933/ using document root <none> and password file <none>
I20260808 18:21:39.757066 22240 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:39.757153 22240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:39.757386 22240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:39.759326 22240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "840bbb2c18da810e77a1b5ac8cd515ff"
server_key_iv: "2a9b4552e6345abf78108f0b79435e07"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.759891 22240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "840bbb2c18da810e77a1b5ac8cd515ff"
server_key_iv: "2a9b4552e6345abf78108f0b79435e07"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.763207 22240 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:21:39.765622 22264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:39.766827 22240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:39.766939 22240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "840bbb2c18da810e77a1b5ac8cd515ff"
server_key_iv: "2a9b4552e6345abf78108f0b79435e07"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.767045 22240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:39.789109 22240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:39.791663 22240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:39.791844 22240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:39.792366 22240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:39.793079 22240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:39.793143 22240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:39.793208 22240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:39.793254 22240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:39.801843 22240 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:34023
I20260808 18:21:39.801853 22433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:34023 every 8 connection(s)
I20260808 18:21:39.802618 22240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:39.808920 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 22240
I20260808 18:21:39.809069 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:21:39.809351 27723 external_mini_cluster.cc:1428] Setting key ae21910632f0ab245d8b9f86a6ff3fd5
Aug 08 18:21:39 dist-test-slave-x4qh krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:39.816326 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:39.804442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:52845 (local address 127.27.18.254:35667)
0808 18:21:39.804592 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:39.804596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:39.805408 (+ 812us) server_negotiation.cc:407] Connection header received
0808 18:21:39.806656 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:39.806660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:39.806716 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:39.806816 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:39.807930 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:39.808458 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:39.809700 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:39.809915 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:39.810021 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:39.812598 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:39.812603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:39.812608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:39.812638 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:39.814100 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:39.814642 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:39.814648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:39.814653 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:39.814747 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:39.815054 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:39.815060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:39.815064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:39.815324 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:39.815452 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:39.815962 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 18:21:39.816094 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:21:39.817042 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:39.804822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35667 (local address 127.27.18.193:52845)
0808 18:21:39.805267 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:39.805307 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:39.806458 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:39.806984 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:39.806991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:39.807295 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:39.807781 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:39.807794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:39.808593 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:39.808597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:39.809585 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:39.809595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:39.810055 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:39.810059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:39.810209 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:39.810736 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:39.810751 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:39.812442 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:39.814230 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:39.814235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:39.814259 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:39.814531 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:39.814842 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:39.814844 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:39.814845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:39.814963 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:39.815461 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:39.815466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:39.815697 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 18:21:39.815873 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 18:21:39.816158 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":130,"threads_started":1}
I20260808 18:21:39.818162 22435 heartbeater.cc:347] Connected to a master server at 127.27.18.254:35667
I20260808 18:21:39.818393 22435 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:39.818827 22435 heartbeater.cc:511] Master 127.27.18.254:35667 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:21:39.820333 22152 ts_manager.cc:194] Registered new tserver with Master: d4dd8e6192ec49f3ad2a4b6974c64d54 (127.27.18.193:34023)
I20260808 18:21:39.821463 22152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:52845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:21:39.837754 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35667
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:39.942888 22441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:39.943107 22441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:39.943143 22441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:39.947021 22441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:39.947081 22441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:39.947154 22441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:39.952033 22441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:39.952950 22441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:39.953872 22441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:39.963016 22455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:39.963032 22452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:39.963022 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:39.963419 22441 server_base.cc:1034] running on GCE node
I20260808 18:21:39.963802 22441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:39.964469 22441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:39.965687 22441 hybrid_clock.cc:648] HybridClock initialized: now 1786213299965658 us; error 37 us; skew 500 ppm
Aug 08 18:21:39 dist-test-slave-x4qh krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:39.968415 22441 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:21:39.969193 22441 webserver.cc:460] Webserver started at http://127.27.18.194:43937/ using document root <none> and password file <none>
I20260808 18:21:39.969730 22441 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:39.969821 22441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:39.970044 22441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:39.972098 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "bbbbd27544f243c5b5c26f9dc35d3e31"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "598affac666ca3fa6dd22ef8c6765594"
server_key_iv: "befcd5c8bd80b765946e4f9bf15c42ad"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.972622 22441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "bbbbd27544f243c5b5c26f9dc35d3e31"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "598affac666ca3fa6dd22ef8c6765594"
server_key_iv: "befcd5c8bd80b765946e4f9bf15c42ad"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.976060 22441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:21:39.978143 22468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:39.979177 22441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:39.979310 22441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "bbbbd27544f243c5b5c26f9dc35d3e31"
format_stamp: "Formatted at 2026-08-08 18:21:39 on dist-test-slave-x4qh"
server_key: "598affac666ca3fa6dd22ef8c6765594"
server_key_iv: "befcd5c8bd80b765946e4f9bf15c42ad"
server_key_version: "encryptionkey@0"
I20260808 18:21:39.979461 22441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:40.007652 22441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:40.010773 22441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:40.010975 22441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:40.011603 22441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:40.012392 22441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:40.012458 22441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:40.012523 22441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:40.012570 22441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:40.022027 22441 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:40437
I20260808 18:21:40.022056 22625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:40437 every 8 connection(s)
I20260808 18:21:40.022881 22441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:40.023826 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 22441
I20260808 18:21:40.023977 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:21:40.024322 27723 external_mini_cluster.cc:1428] Setting key 73a0d5864c4689d047f804d2ec5c7fbe
Aug 08 18:21:40 dist-test-slave-x4qh krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213299, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
I20260808 18:21:40.036695 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.024814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:39951 (local address 127.27.18.254:35667)
0808 18:21:40.024970 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.024974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.025803 (+ 829us) server_negotiation.cc:407] Connection header received
0808 18:21:40.027079 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.027082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.027135 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.027238 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:40.028411 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.029070 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.030216 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.030445 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.030556 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.033315 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:40.033318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:40.033320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:40.033342 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:40.034831 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:40.035247 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:40.035249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:40.035250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:40.035374 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:40.035727 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:40.035729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:40.035730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:40.035907 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:40.035981 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.036476 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.036583 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
I20260808 18:21:40.037461 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.025206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35667 (local address 127.27.18.194:39951)
0808 18:21:40.025667 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.025717 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.026871 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.027465 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.027471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.027788 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:40.028258 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.028272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.029199 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.029202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.030078 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.030087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.030597 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.030601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.030763 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.031345 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:40.031373 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:40.033126 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:40.034935 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:40.034939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:40.034950 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:40.035167 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:40.035505 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:40.035508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:40.035509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:40.035627 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:40.035994 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:40.035998 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:40.036207 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.036361 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.036594 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":133,"threads_started":1}
I20260808 18:21:40.038638 22627 heartbeater.cc:347] Connected to a master server at 127.27.18.254:35667
I20260808 18:21:40.038903 22627 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:40.039511 22627 heartbeater.cc:511] Master 127.27.18.254:35667 requested a full tablet report, sending...
I20260808 18:21:40.040673 22152 ts_manager.cc:194] Registered new tserver with Master: bbbbd27544f243c5b5c26f9dc35d3e31 (127.27.18.194:40437)
I20260808 18:21:40.041563 22152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:39951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:21:40.052017 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35667
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:40.161377 22633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:40.161594 22633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:40.161635 22633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:40.165572 22633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:40.165616 22633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:40.165704 22633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:40.170567 22633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:35667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:40.171473 22633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:40.172412 22633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:40.181646 22644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:40.181694 22633 server_base.cc:1034] running on GCE node
W20260808 18:21:40.181612 22643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:40.181630 22646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:40.182245 22633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:40.182765 22633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:40.183948 22633 hybrid_clock.cc:648] HybridClock initialized: now 1786213300183923 us; error 41 us; skew 500 ppm
Aug 08 18:21:40 dist-test-slave-x4qh krb5kdc[22086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213300, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:40.186486 22633 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:21:40.187330 22633 webserver.cc:460] Webserver started at http://127.27.18.195:41245/ using document root <none> and password file <none>
I20260808 18:21:40.187858 22633 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:40.187951 22633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:40.188191 22633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:40.190064 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "9f38dd730b4844909fd27b6ff9516aa4"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "97efb31bc6df737ced6d3e6ba1de6a53"
server_key_iv: "b81c096ddc665c7b5965ece346286cc0"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.190661 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "9f38dd730b4844909fd27b6ff9516aa4"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "97efb31bc6df737ced6d3e6ba1de6a53"
server_key_iv: "b81c096ddc665c7b5965ece346286cc0"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.194135 22633 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:21:40.196436 22655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:40.197458 22633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:21:40.197589 22633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "9f38dd730b4844909fd27b6ff9516aa4"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "97efb31bc6df737ced6d3e6ba1de6a53"
server_key_iv: "b81c096ddc665c7b5965ece346286cc0"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.197701 22633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:40.220108 22633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:40.222800 22633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:40.222995 22633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:40.223574 22633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:40.224467 22633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:40.224560 22633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:40.224632 22633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:40.224699 22633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:40.234071 22633 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:43759
I20260808 18:21:40.234094 22813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:43759 every 8 connection(s)
I20260808 18:21:40.234895 22633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:40.238011 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 22633
I20260808 18:21:40.238152 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:21:40.238471 27723 external_mini_cluster.cc:1428] Setting key bdc59931ecf55956c74714418bf44079
Aug 08 18:21:40 dist-test-slave-x4qh krb5kdc[22086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213300, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:40.247864 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.236566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:40479 (local address 127.27.18.254:35667)
0808 18:21:40.236715 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.236718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.237424 (+ 706us) server_negotiation.cc:407] Connection header received
0808 18:21:40.238972 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.238975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.239034 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.239139 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:40.240339 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.240890 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.241915 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.242125 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.242227 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.244523 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:40.244533 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:40.244536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:40.244560 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:40.245893 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:40.246375 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:40.246395 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:40.246396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:40.246466 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:40.246791 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:40.246796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:40.246797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:40.246961 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:40.247065 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.247572 (+ 507us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.247695 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:21:40.248507 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.236879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:35667 (local address 127.27.18.195:40479)
0808 18:21:40.237293 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.237329 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.238754 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.239312 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.239318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.239788 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:40.240204 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.240217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.241008 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.241012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.241793 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.241802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.242260 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.242263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.242409 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.242938 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:40.242953 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:40.244367 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:40.246011 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:40.246017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:40.246029 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:40.246271 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:40.246563 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:40.246565 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:40.246567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:40.246698 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:40.247065 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:40.247071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:40.247290 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.247506 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.247794 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":110,"threads_started":1}
I20260808 18:21:40.249518 22815 heartbeater.cc:347] Connected to a master server at 127.27.18.254:35667
I20260808 18:21:40.249749 22815 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:40.250191 22815 heartbeater.cc:511] Master 127.27.18.254:35667 requested a full tablet report, sending...
I20260808 18:21:40.251188 22152 ts_manager.cc:194] Registered new tserver with Master: 9f38dd730b4844909fd27b6ff9516aa4 (127.27.18.195:43759)
I20260808 18:21:40.251821 22152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:40479
I20260808 18:21:40.252388 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:40.260782 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.254316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.27.18.254:35667)
0808 18:21:40.254500 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.254505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.254573 (+ 68us) server_negotiation.cc:407] Connection header received
0808 18:21:40.254697 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.254699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.254736 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.254818 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:40.255502 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.255966 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.256717 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.256912 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.256991 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.258235 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:40.258238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:40.258240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:40.258261 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:40.259504 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:40.259849 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:40.259851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:40.259852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:40.259940 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:40.260157 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:40.260159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:40.260160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:40.260331 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:40.260426 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.260604 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.260689 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 18:21:40.263610 22151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:40.266614 22151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:40.281004 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.275651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.27.18.195:43759)
0808 18:21:40.276225 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.276236 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.276447 (+ 211us) server_negotiation.cc:407] Connection header received
0808 18:21:40.276520 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.276524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.276714 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.276861 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:40.277089 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.278118 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.279866 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.280378 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.280496 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.280641 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.280781 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.280866 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":139,"threads_started":1}
I20260808 18:21:40.280995 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.275459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:43759 (local address 127.0.0.1:46068)
0808 18:21:40.276257 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.276269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.276353 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.276860 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.276863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.276880 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:40.276970 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.276977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.278241 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.278244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.279716 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.279728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.280498 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.280502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.280599 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.280650 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.280754 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.280867 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":241,"thread_start_us":86,"threads_started":1}
I20260808 18:21:40.281273 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.274861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.27.18.193:34023)
0808 18:21:40.275134 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.275149 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.275775 (+ 626us) server_negotiation.cc:407] Connection header received
0808 18:21:40.276076 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.276083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.276311 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.276442 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:40.276924 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.278090 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.279640 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.280439 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.280543 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.280718 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.281033 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.281133 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260808 18:21:40.281265 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.274780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:40437 (local address 127.0.0.1:51584)
0808 18:21:40.275834 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.275848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.275943 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.277016 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.277021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.277037 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:40.277123 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.277128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.278084 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.278089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.279716 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.279728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.280898 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.280901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.280998 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.281017 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.281077 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.281138 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":119,"thread_start_us":95,"threads_started":1}
I20260808 18:21:40.281591 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.274741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:34023 (local address 127.0.0.1:53222)
0808 18:21:40.275638 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.275683 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.275889 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.276646 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.276650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.276669 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:40.276791 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.276799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.278268 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.278271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.279521 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.279531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.280783 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.280787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.280881 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.280902 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.281412 (+ 510us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.281479 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":77,"thread_start_us":93,"threads_started":1}
I20260808 18:21:40.281651 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.275723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.27.18.194:40437)
0808 18:21:40.276428 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.276439 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.276463 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:21:40.276540 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.276559 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.276731 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.276881 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:40.277248 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.277944 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.279846 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.280603 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.280724 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.280845 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.281442 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.281531 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":124,"threads_started":1}
I20260808 18:21:40.284350 22535 tablet_service.cc:1467] Processing CreateTablet for tablet ea02c3c887694d1b80fd43208ada1b45 (DEFAULT_TABLE table=test-table [id=4f40a062388e452b86f5346a9357ca44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:40.284435 22723 tablet_service.cc:1467] Processing CreateTablet for tablet ea02c3c887694d1b80fd43208ada1b45 (DEFAULT_TABLE table=test-table [id=4f40a062388e452b86f5346a9357ca44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:40.284435 22334 tablet_service.cc:1467] Processing CreateTablet for tablet ea02c3c887694d1b80fd43208ada1b45 (DEFAULT_TABLE table=test-table [id=4f40a062388e452b86f5346a9357ca44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:40.285341 22535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea02c3c887694d1b80fd43208ada1b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:40.285426 22723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea02c3c887694d1b80fd43208ada1b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:40.285426 22334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea02c3c887694d1b80fd43208ada1b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:40.291648 22839 tablet_bootstrap.cc:492] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Bootstrap starting.
I20260808 18:21:40.292608 22840 tablet_bootstrap.cc:492] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Bootstrap starting.
I20260808 18:21:40.292971 22841 tablet_bootstrap.cc:492] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: Bootstrap starting.
I20260808 18:21:40.293721 22839 tablet_bootstrap.cc:654] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:40.294638 22841 tablet_bootstrap.cc:654] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:40.294806 22839 log.cc:824] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:40.295924 22841 log.cc:824] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:40.296903 22839 tablet_bootstrap.cc:492] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: No bootstrap required, opened a new log
I20260808 18:21:40.297046 22840 tablet_bootstrap.cc:654] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:40.297124 22839 ts_tablet_manager.cc:1397] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 18:21:40.297904 22841 tablet_bootstrap.cc:492] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: No bootstrap required, opened a new log
I20260808 18:21:40.298079 22841 ts_tablet_manager.cc:1397] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:21:40.298211 22840 log.cc:824] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:40.300107 22840 tablet_bootstrap.cc:492] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: No bootstrap required, opened a new log
I20260808 18:21:40.300269 22840 ts_tablet_manager.cc:1397] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 18:21:40.300511 22839 raft_consensus.cc:348] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.300766 22839 raft_consensus.cc:374] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:40.300854 22839 raft_consensus.cc:729] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbbbd27544f243c5b5c26f9dc35d3e31, State: Initialized, Role: FOLLOWER
I20260808 18:21:40.301378 22839 consensus_queue.cc:260] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.301771 22841 raft_consensus.cc:348] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.302038 22627 heartbeater.cc:503] Master 127.27.18.254:35667 was elected leader, sending a full tablet report...
I20260808 18:21:40.302042 22841 raft_consensus.cc:374] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:40.302102 22841 raft_consensus.cc:729] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4dd8e6192ec49f3ad2a4b6974c64d54, State: Initialized, Role: FOLLOWER
I20260808 18:21:40.302424 22839 ts_tablet_manager.cc:1428] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:21:40.302919 22841 consensus_queue.cc:260] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.303679 22435 heartbeater.cc:503] Master 127.27.18.254:35667 was elected leader, sending a full tablet report...
I20260808 18:21:40.303851 22841 ts_tablet_manager.cc:1428] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 18:21:40.304107 22436 tablet.cc:2369] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:40.303978 22840 raft_consensus.cc:348] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.304198 22840 raft_consensus.cc:374] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:40.304235 22840 raft_consensus.cc:729] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f38dd730b4844909fd27b6ff9516aa4, State: Initialized, Role: FOLLOWER
I20260808 18:21:40.304574 22840 consensus_queue.cc:260] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.305179 22815 heartbeater.cc:503] Master 127.27.18.254:35667 was elected leader, sending a full tablet report...
I20260808 18:21:40.305472 22840 ts_tablet_manager.cc:1428] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:21:40.482215 22848 raft_consensus.cc:482] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:40.482463 22848 raft_consensus.cc:504] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.483793 22848 leader_election.cc:288] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4dd8e6192ec49f3ad2a4b6974c64d54 (127.27.18.193:34023), bbbbd27544f243c5b5c26f9dc35d3e31 (127.27.18.194:40437)
W20260808 18:21:40.485836 22816 tablet.cc:2369] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:40.487828 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.484208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:48385 (local address 127.27.18.194:40437)
0808 18:21:40.484337 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.484360 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.484553 (+ 193us) server_negotiation.cc:407] Connection header received
0808 18:21:40.484657 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.484660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.484718 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.484787 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:40.485067 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.485790 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.486854 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.487246 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.487379 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.487450 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.487648 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.487721 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 18:21:40.487846 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.484055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:40437 (local address 127.27.18.195:48385)
0808 18:21:40.484468 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.484479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.484555 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.484821 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.484824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.484838 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:40.484966 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.484971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.485878 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.485882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.486731 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.486741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.487376 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.487405 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.487513 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.487542 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.487631 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.487699 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":159,"threads_started":1}
I20260808 18:21:40.488030 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.483980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:34023 (local address 127.27.18.195:54069)
0808 18:21:40.484179 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.484197 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:40.484310 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:40.484650 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:40.484660 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:40.484684 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:40.484822 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.484828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.485832 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.485835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.486796 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:40.486806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.487648 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.487651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:40.487735 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.487752 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:21:40.487843 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 18:21:40.487912 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":68}
I20260808 18:21:40.488046 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.484047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:54069 (local address 127.27.18.193:34023)
0808 18:21:40.484192 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.484198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.484265 (+ 67us) server_negotiation.cc:407] Connection header received
0808 18:21:40.484463 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.484465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.484514 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.484622 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:40.484949 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.485720 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.486919 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.487325 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.487487 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.487546 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.487864 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.487943 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:21:40.488620 22560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02c3c887694d1b80fd43208ada1b45" candidate_uuid: "9f38dd730b4844909fd27b6ff9516aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" is_pre_election: true
I20260808 18:21:40.488818 22366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02c3c887694d1b80fd43208ada1b45" candidate_uuid: "9f38dd730b4844909fd27b6ff9516aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" is_pre_election: true
I20260808 18:21:40.488952 22560 raft_consensus.cc:2454] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f38dd730b4844909fd27b6ff9516aa4 in term 0.
I20260808 18:21:40.489097 22366 raft_consensus.cc:2454] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f38dd730b4844909fd27b6ff9516aa4 in term 0.
I20260808 18:21:40.489430 22660 leader_election.cc:302] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f38dd730b4844909fd27b6ff9516aa4, bbbbd27544f243c5b5c26f9dc35d3e31; no voters:
I20260808 18:21:40.489711 22848 raft_consensus.cc:2781] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:40.489784 22848 raft_consensus.cc:482] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:40.489811 22848 raft_consensus.cc:3037] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:40.490810 22848 raft_consensus.cc:504] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.491148 22848 leader_election.cc:288] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [CANDIDATE]: Term 1 election: Requested vote from peers d4dd8e6192ec49f3ad2a4b6974c64d54 (127.27.18.193:34023), bbbbd27544f243c5b5c26f9dc35d3e31 (127.27.18.194:40437)
I20260808 18:21:40.491483 22366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02c3c887694d1b80fd43208ada1b45" candidate_uuid: "9f38dd730b4844909fd27b6ff9516aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54"
I20260808 18:21:40.491603 22560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea02c3c887694d1b80fd43208ada1b45" candidate_uuid: "9f38dd730b4844909fd27b6ff9516aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31"
I20260808 18:21:40.491611 22366 raft_consensus.cc:3037] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:40.491748 22560 raft_consensus.cc:3037] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:40.492622 22366 raft_consensus.cc:2454] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f38dd730b4844909fd27b6ff9516aa4 in term 1.
I20260808 18:21:40.492844 22560 raft_consensus.cc:2454] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f38dd730b4844909fd27b6ff9516aa4 in term 1.
I20260808 18:21:40.492955 22659 leader_election.cc:302] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f38dd730b4844909fd27b6ff9516aa4, d4dd8e6192ec49f3ad2a4b6974c64d54; no voters:
I20260808 18:21:40.493142 22848 raft_consensus.cc:2781] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:40.493359 22848 raft_consensus.cc:686] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 1 LEADER]: Becoming Leader. State: Replica: 9f38dd730b4844909fd27b6ff9516aa4, State: Running, Role: LEADER
I20260808 18:21:40.493710 22848 consensus_queue.cc:237] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } }
I20260808 18:21:40.496822 22152 catalog_manager.cc:5495] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f38dd730b4844909fd27b6ff9516aa4 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "9f38dd730b4844909fd27b6ff9516aa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f38dd730b4844909fd27b6ff9516aa4" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 43759 } health_report { overall_health: HEALTHY } } }
I20260808 18:21:40.511230 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.508086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.27.18.195:43759)
0808 18:21:40.508211 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.508215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.508227 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:21:40.508353 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.508355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.508404 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.508499 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:40.508705 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.509148 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.509969 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.510184 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.510277 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.510526 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:40.510893 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:40.510987 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.511056 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.511124 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:21:40.516467 22560 raft_consensus.cc:1261] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 1 FOLLOWER]: Refusing update from remote peer 9f38dd730b4844909fd27b6ff9516aa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:40.516474 22366 raft_consensus.cc:1261] T ea02c3c887694d1b80fd43208ada1b45 P d4dd8e6192ec49f3ad2a4b6974c64d54 [term 1 FOLLOWER]: Refusing update from remote peer 9f38dd730b4844909fd27b6ff9516aa4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:40.517118 22851 consensus_queue.cc:1035] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbbbd27544f243c5b5c26f9dc35d3e31" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 40437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:40.517299 22848 consensus_queue.cc:1035] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4dd8e6192ec49f3ad2a4b6974c64d54" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:40.523818 22855 mvcc.cc:205] Tried to move back new op lower bound from 7316329678909018112 to 7316329678824804352. Current Snapshot: MvccSnapshot[applied={T|T < 7316329678909018112}]
W20260808 18:21:40.523984 22629 tablet.cc:2369] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:40.525308 22856 mvcc.cc:205] Tried to move back new op lower bound from 7316329678909018112 to 7316329678824804352. Current Snapshot: MvccSnapshot[applied={T|T < 7316329678909018112}]
I20260808 18:21:40.534771 22151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:40.534910 22151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:40.537317 22151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a28de6412494ba3b35fd64e55a1bc04: Sending DeleteTablet for 3 replicas of tablet ea02c3c887694d1b80fd43208ada1b45
I20260808 18:21:40.538031 22334 tablet_service.cc:1514] Processing DeleteTablet for tablet ea02c3c887694d1b80fd43208ada1b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:40 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:53222
I20260808 18:21:40.538146 22723 tablet_service.cc:1514] Processing DeleteTablet for tablet ea02c3c887694d1b80fd43208ada1b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:40 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:46068
I20260808 18:21:40.538319 22535 tablet_service.cc:1514] Processing DeleteTablet for tablet ea02c3c887694d1b80fd43208ada1b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:40 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:51584
I20260808 18:21:40.538637 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 22240
I20260808 18:21:40.551904 22870 tablet_replica.cc:330] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: stopping tablet replica
I20260808 18:21:40.552210 22869 tablet_replica.cc:330] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: stopping tablet replica
I20260808 18:21:40.552258 22870 raft_consensus.cc:2229] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:40.552551 22869 raft_consensus.cc:2229] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:21:40.552577 22870 raft_consensus.cc:2258] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:40.553030 22869 raft_consensus.cc:2258] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:40.554576 22869 ts_tablet_manager.cc:1905] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:40.555676 22870 ts_tablet_manager.cc:1905] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:40.559434 22870 ts_tablet_manager.cc:1918] T ea02c3c887694d1b80fd43208ada1b45 P bbbbd27544f243c5b5c26f9dc35d3e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260808 18:21:40.561261 22132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:34023 (error 108)
I20260808 18:21:40.562319 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.562030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:34023 (local address 127.0.0.1:53226)
0808 18:21:40.562172 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.562244 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:34023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 18:21:40.562490 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 22441
W20260808 18:21:40.562587 22132 catalog_manager.cc:4584] TS d4dd8e6192ec49f3ad2a4b6974c64d54 (127.27.18.193:34023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea02c3c887694d1b80fd43208ada1b45: Network error: Client connection negotiation failed: client connection to 127.27.18.193:34023: connect: Connection refused (error 111)
I20260808 18:21:40.562856 22869 ts_tablet_manager.cc:1918] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:40.562948 22869 log.cc:1196] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/ea02c3c887694d1b80fd43208ada1b45
I20260808 18:21:40.564157 22869 ts_tablet_manager.cc:1939] T ea02c3c887694d1b80fd43208ada1b45 P 9f38dd730b4844909fd27b6ff9516aa4: Deleting consensus metadata
I20260808 18:21:40.565577 22133 catalog_manager.cc:4841] TS 9f38dd730b4844909fd27b6ff9516aa4 (127.27.18.195:43759): tablet ea02c3c887694d1b80fd43208ada1b45 (table test-table [id=4f40a062388e452b86f5346a9357ca44]) successfully deleted
I20260808 18:21:40.572615 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 22633
I20260808 18:21:40.572805 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.572482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:40437 (local address 127.0.0.1:51590)
0808 18:21:40.572641 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:40.572712 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.194:40437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 18:21:40.583081 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 22105
2026-08-08T18:21:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2496 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T18:21:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:40Z Disabled control of system clock
I20260808 18:21:40.617679 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36489
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:36489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:21:40.721642 22878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:40.721839 22878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:40.721869 22878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:40.725786 22878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:40.725835 22878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:40.725852 22878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:40.725867 22878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:40.725883 22878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:40.730301 22878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:36489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:40.731071 22878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:40.732299 22878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:40.740119 22890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:40.740152 22878 server_base.cc:1034] running on GCE node
W20260808 18:21:40.740119 22885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:40.740108 22886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:40.740932 22878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:40.741925 22878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:40.743110 22878 hybrid_clock.cc:648] HybridClock initialized: now 1786213300743099 us; error 48 us; skew 500 ppm
I20260808 18:21:40.744830 22878 webserver.cc:460] Webserver started at http://127.27.18.254:42687/ using document root <none> and password file <none>
I20260808 18:21:40.745364 22878 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:40.745453 22878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:40.745656 22878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:40.747588 22878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "81bee03854a6441b97a6598de17e7a87"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "3c9a99915edf7e7a26f572ac47a7e564"
server_key_iv: "23966af1abe993d06d912b8093c22266"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.748121 22878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "81bee03854a6441b97a6598de17e7a87"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "3c9a99915edf7e7a26f572ac47a7e564"
server_key_iv: "23966af1abe993d06d912b8093c22266"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.751607 22878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:21:40.753904 22899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:40.755074 22878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:40.755236 22878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "81bee03854a6441b97a6598de17e7a87"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "3c9a99915edf7e7a26f572ac47a7e564"
server_key_iv: "23966af1abe993d06d912b8093c22266"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.755345 22878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:40.775331 22878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:40.776000 22878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:40.776158 22878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:40.784160 22878 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:36489
I20260808 18:21:40.784199 22967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:36489 every 8 connection(s)
I20260808 18:21:40.785032 22878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:40.787865 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:40.790865 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 22878
I20260808 18:21:40.790984 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:40.791306 27723 external_mini_cluster.cc:1428] Setting key 16b0b3bb74f554500cdf58866d8dcf4e
I20260808 18:21:40.793195 22968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87: Bootstrap starting.
I20260808 18:21:40.795203 22968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:40.795912 22968 log.cc:824] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:40.797835 22968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87: No bootstrap required, opened a new log
I20260808 18:21:40.800108 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:40.792654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.27.18.254:36489)
0808 18:21:40.793219 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:40.793232 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:40.793264 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:21:40.793891 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:40.793907 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:40.794277 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:40.794608 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:40.795018 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.795979 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.797078 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:40.797497 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:40.797606 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:40.798475 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:40.798480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:40.798500 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:40.798856 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:40.798909 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:40.798937 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:40.799052 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:40.799221 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 18:21:40.799506 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260808 18:21:40.801622 22968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bee03854a6441b97a6598de17e7a87" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36489 } }
I20260808 18:21:40.801856 22968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:40.801932 22968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81bee03854a6441b97a6598de17e7a87, State: Initialized, Role: FOLLOWER
I20260808 18:21:40.802389 22968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bee03854a6441b97a6598de17e7a87" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36489 } }
I20260808 18:21:40.802536 22968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:40.802623 22968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:40.802719 22968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:40.803601 22968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bee03854a6441b97a6598de17e7a87" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36489 } }
I20260808 18:21:40.803933 22968 leader_election.cc:302] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81bee03854a6441b97a6598de17e7a87; no voters:
I20260808 18:21:40.804226 22968 leader_election.cc:288] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:40.804374 22974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:40.804605 22974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [term 1 LEADER]: Becoming Leader. State: Replica: 81bee03854a6441b97a6598de17e7a87, State: Running, Role: LEADER
I20260808 18:21:40.804930 22974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bee03854a6441b97a6598de17e7a87" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36489 } }
I20260808 18:21:40.805348 22968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:40.806417 22975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81bee03854a6441b97a6598de17e7a87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bee03854a6441b97a6598de17e7a87" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36489 } } }
I20260808 18:21:40.806551 22975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:40.807133 22987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:40.806857 22977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81bee03854a6441b97a6598de17e7a87. Latest consensus state: current_term: 1 leader_uuid: "81bee03854a6441b97a6598de17e7a87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bee03854a6441b97a6598de17e7a87" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36489 } } }
I20260808 18:21:40.807652 22977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:40.810339 22987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:40.815302 22987 catalog_manager.cc:1347] Generated new cluster ID: f743d37a27044f398873c4bdb3b8f344
I20260808 18:21:40.815361 22987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:40.828166 22987 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:40.829147 22987 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:40.838908 22987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87: Generated new TSK 0
I20260808 18:21:40.839599 22987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:40.849548 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36489
--builtin_ntp_servers=127.27.18.212:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:21:40.972147 23000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:40.972348 23000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:40.972381 23000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:40.976033 23000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:40.976075 23000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:40.976154 23000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:40.980329 23000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:40.981195 23000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:40.982355 23000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:21:40.991230 23000 server_base.cc:1034] running on GCE node
W20260808 18:21:40.991241 23008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:40.991187 23010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:40.991394 23012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:40.991717 23000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:40.992307 23000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:40.993501 23000 hybrid_clock.cc:648] HybridClock initialized: now 1786213300993458 us; error 37 us; skew 500 ppm
I20260808 18:21:40.995119 23000 webserver.cc:460] Webserver started at http://127.27.18.193:43379/ using document root <none> and password file <none>
I20260808 18:21:40.995689 23000 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:40.995774 23000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:40.996008 23000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:40.997749 23000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "c05a23ae59da48818c5296f95690e1a4"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "efb1f0a6f7ebf7be23aefbc538275736"
server_key_iv: "3b358f450051e42a6900f51bb3fe102b"
server_key_version: "encryptionkey@0"
I20260808 18:21:40.998297 23000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "c05a23ae59da48818c5296f95690e1a4"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "efb1f0a6f7ebf7be23aefbc538275736"
server_key_iv: "3b358f450051e42a6900f51bb3fe102b"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.001617 23000 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:21:41.003732 23019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.004910 23000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:41.005034 23000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "c05a23ae59da48818c5296f95690e1a4"
format_stamp: "Formatted at 2026-08-08 18:21:40 on dist-test-slave-x4qh"
server_key: "efb1f0a6f7ebf7be23aefbc538275736"
server_key_iv: "3b358f450051e42a6900f51bb3fe102b"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.005141 23000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:41.025462 23000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:41.026005 23000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:41.026183 23000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:41.026741 23000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:41.027647 23000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:41.027706 23000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.027751 23000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:41.027779 23000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.036336 23000 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:45395
I20260808 18:21:41.036360 23177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:45395 every 8 connection(s)
I20260808 18:21:41.037164 23000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:41.045346 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.038805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:38223 (local address 127.27.18.254:36489)
0808 18:21:41.038940 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.038943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.039853 (+ 910us) server_negotiation.cc:407] Connection header received
0808 18:21:41.041121 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.041123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.041172 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.041238 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:41.041969 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.042494 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.043620 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.043853 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.043935 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.044957 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:41.044960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:41.044962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:41.044973 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:41.044988 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:41.044994 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:41.045047 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.045191 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.045239 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:21:41.046062 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.039080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36489 (local address 127.27.18.193:38223)
0808 18:21:41.039732 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.039766 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.040954 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.041381 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.041387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.041454 (+ 67us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:41.041860 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.041871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.042598 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.042601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.043489 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.043499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.043982 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.043984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.044119 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.044733 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:41.044734 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:41.044769 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:41.044832 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:41.045066 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:41.045073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:41.045077 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.045162 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.045349 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":108,"threads_started":1}
I20260808 18:21:41.046958 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 23000
I20260808 18:21:41.047093 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:21:41.047312 23179 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36489
I20260808 18:21:41.047449 27723 external_mini_cluster.cc:1428] Setting key c59bda8cddc1dd940984d1ef120d7d1c
I20260808 18:21:41.047540 23179 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:41.048029 23179 heartbeater.cc:511] Master 127.27.18.254:36489 requested a full tablet report, sending...
I20260808 18:21:41.049355 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36489
--builtin_ntp_servers=127.27.18.212:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:21:41.049407 22918 ts_manager.cc:194] Registered new tserver with Master: c05a23ae59da48818c5296f95690e1a4 (127.27.18.193:45395)
I20260808 18:21:41.050457 22918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.193:38223
W20260808 18:21:41.152475 23182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:41.152678 23182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:41.152710 23182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:41.156350 23182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:41.156390 23182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:41.156471 23182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:41.160761 23182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:41.161640 23182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:41.162818 23182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:21:41.171468 23182 server_base.cc:1034] running on GCE node
W20260808 18:21:41.171482 23191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:41.171515 23192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:41.171605 23195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:41.172015 23182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:41.172624 23182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:41.173803 23182 hybrid_clock.cc:648] HybridClock initialized: now 1786213301173785 us; error 28 us; skew 500 ppm
I20260808 18:21:41.175406 23182 webserver.cc:460] Webserver started at http://127.27.18.194:45735/ using document root <none> and password file <none>
I20260808 18:21:41.175932 23182 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:41.176020 23182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:41.176214 23182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:41.177922 23182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "eb68f0168d7c4d7698f225258a758ed4"
format_stamp: "Formatted at 2026-08-08 18:21:41 on dist-test-slave-x4qh"
server_key: "a80dd99f1ff82b0db56d06ddcb87396e"
server_key_iv: "5a18a3a60e2424e6281b7196cb3a1b4a"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.178409 23182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "eb68f0168d7c4d7698f225258a758ed4"
format_stamp: "Formatted at 2026-08-08 18:21:41 on dist-test-slave-x4qh"
server_key: "a80dd99f1ff82b0db56d06ddcb87396e"
server_key_iv: "5a18a3a60e2424e6281b7196cb3a1b4a"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.181679 23182 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 18:21:41.183828 23204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.185048 23182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:41.185163 23182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "eb68f0168d7c4d7698f225258a758ed4"
format_stamp: "Formatted at 2026-08-08 18:21:41 on dist-test-slave-x4qh"
server_key: "a80dd99f1ff82b0db56d06ddcb87396e"
server_key_iv: "5a18a3a60e2424e6281b7196cb3a1b4a"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.185273 23182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:41.196637 23182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:41.197155 23182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:41.197360 23182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:41.197882 23182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:41.198694 23182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:41.198771 23182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.198833 23182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:41.198887 23182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.207687 23182 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:43025
I20260808 18:21:41.207722 23377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:43025 every 8 connection(s)
I20260808 18:21:41.208410 23182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:41.216027 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 23182
I20260808 18:21:41.216159 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:21:41.216341 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.210021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:56175 (local address 127.27.18.254:36489)
0808 18:21:41.210141 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.210145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.210972 (+ 827us) server_negotiation.cc:407] Connection header received
0808 18:21:41.212161 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.212163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.212207 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.212274 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:41.213009 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.213502 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.214562 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.214793 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.214860 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.215832 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:41.215835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:41.215837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:41.215847 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:41.215862 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:41.215868 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:41.215920 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.216116 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.216184 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:21:41.216456 27723 external_mini_cluster.cc:1428] Setting key 8227f3b535d201279f472cf7e1ad1344
I20260808 18:21:41.217082 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.210276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36489 (local address 127.27.18.194:56175)
0808 18:21:41.210831 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.210871 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.211998 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.212408 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.212414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.212481 (+ 67us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:41.212869 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.212898 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.213631 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.213634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.214458 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.214466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.214906 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.214908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.215035 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.215598 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:41.215599 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:41.215632 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:41.215683 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:41.215951 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:41.215960 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:41.215966 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.216092 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.216345 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
I20260808 18:21:41.218035 23378 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36489
I20260808 18:21:41.218248 23378 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:41.218401 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36489
--builtin_ntp_servers=127.27.18.212:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:21:41.218742 23378 heartbeater.cc:511] Master 127.27.18.254:36489 requested a full tablet report, sending...
I20260808 18:21:41.219668 22918 ts_manager.cc:194] Registered new tserver with Master: eb68f0168d7c4d7698f225258a758ed4 (127.27.18.194:43025)
I20260808 18:21:41.220191 22918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.194:56175
W20260808 18:21:41.322549 23382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:41.322791 23382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:41.322871 23382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:41.326655 23382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:41.326730 23382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:41.326825 23382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:41.331194 23382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:41.332082 23382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:41.333282 23382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:41.342602 23389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:41.342631 23388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:41.342698 23382 server_base.cc:1034] running on GCE node
W20260808 18:21:41.342650 23391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:41.343124 23382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:41.343783 23382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:41.344973 23382 hybrid_clock.cc:648] HybridClock initialized: now 1786213301344956 us; error 28 us; skew 500 ppm
I20260808 18:21:41.346643 23382 webserver.cc:460] Webserver started at http://127.27.18.195:35671/ using document root <none> and password file <none>
I20260808 18:21:41.347180 23382 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:41.347272 23382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:41.347537 23382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:41.349365 23382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "da1cc5e2fcb441d08d2b355283eb653c"
format_stamp: "Formatted at 2026-08-08 18:21:41 on dist-test-slave-x4qh"
server_key: "66d55ff0e4fe2d1c4c2c0f3f5b8f9168"
server_key_iv: "42b0ff200e9f2cd6a6480b7bbbd3056e"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.349893 23382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "da1cc5e2fcb441d08d2b355283eb653c"
format_stamp: "Formatted at 2026-08-08 18:21:41 on dist-test-slave-x4qh"
server_key: "66d55ff0e4fe2d1c4c2c0f3f5b8f9168"
server_key_iv: "42b0ff200e9f2cd6a6480b7bbbd3056e"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.353145 23382 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 18:21:41.355294 23401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.356383 23382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:41.356519 23382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "da1cc5e2fcb441d08d2b355283eb653c"
format_stamp: "Formatted at 2026-08-08 18:21:41 on dist-test-slave-x4qh"
server_key: "66d55ff0e4fe2d1c4c2c0f3f5b8f9168"
server_key_iv: "42b0ff200e9f2cd6a6480b7bbbd3056e"
server_key_version: "encryptionkey@0"
I20260808 18:21:41.356624 23382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:41.370721 23382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:41.371304 23382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:41.371505 23382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:41.372025 23382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:41.372859 23382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:41.372925 23382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.372991 23382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:41.373044 23382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:41.381343 23382 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42003
I20260808 18:21:41.381371 23579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42003 every 8 connection(s)
I20260808 18:21:41.382083 23382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:41.385341 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 23382
I20260808 18:21:41.385463 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:21:41.385746 27723 external_mini_cluster.cc:1428] Setting key 4cff75daced407366606251571a5bb42
I20260808 18:21:41.391273 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.383836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:45077 (local address 127.27.18.254:36489)
0808 18:21:41.383974 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.383977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.384928 (+ 951us) server_negotiation.cc:407] Connection header received
0808 18:21:41.386582 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.386585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.386640 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.386740 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:41.387602 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.388271 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.389444 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.389693 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.389803 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.390776 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:41.390780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:41.390783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:41.390804 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:41.390831 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:41.390839 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:41.390918 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.391079 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.391138 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:21:41.391995 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.384183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36489 (local address 127.27.18.195:45077)
0808 18:21:41.384795 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.384831 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.386389 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.386897 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.386904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.386977 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:41.387457 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.387471 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.388393 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.388396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.389315 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.389325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.389844 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.389848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.390004 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.390537 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:41.390538 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:41.390574 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:41.390630 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:41.390920 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:41.390932 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:41.390939 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.391065 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.391324 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":122,"threads_started":1}
I20260808 18:21:41.392938 23580 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36489
I20260808 18:21:41.393160 23580 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:41.393639 23580 heartbeater.cc:511] Master 127.27.18.254:36489 requested a full tablet report, sending...
I20260808 18:21:41.394575 22918 ts_manager.cc:194] Registered new tserver with Master: da1cc5e2fcb441d08d2b355283eb653c (127.27.18.195:42003)
I20260808 18:21:41.395099 22918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.195:45077
I20260808 18:21:41.399361 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:41.404414 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.401157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.27.18.254:36489)
0808 18:21:41.401293 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.401298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.401512 (+ 214us) server_negotiation.cc:407] Connection header received
0808 18:21:41.401598 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.401600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.401633 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.401694 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:41.401937 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.402368 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.403127 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.403302 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.403400 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.404109 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:41.404112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:41.404113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:41.404122 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:41.404134 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:41.404140 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:41.404192 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.404283 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.404323 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 18:21:41.406827 22918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:41.409982 22918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:41.423310 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.418610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.27.18.195:42003)
0808 18:21:41.419574 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.419585 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.419602 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:21:41.419653 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.419656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.419756 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.419852 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:41.420063 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.420873 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.422144 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.422643 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.422766 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.422929 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.423053 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.423140 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":117,"threads_started":1}
I20260808 18:21:41.423305 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.417474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:45395 (local address 127.0.0.1:58486)
0808 18:21:41.417914 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.417958 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.418068 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.419304 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.419309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.419338 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:41.419488 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.419496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.420530 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.420533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.421860 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.421871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.422844 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.422847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.422928 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.422949 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.423008 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.423142 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":27,"spinlock_wait_cycles":2304,"thread_start_us":72,"threads_started":1}
I20260808 18:21:41.423321 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.417579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58486 (local address 127.27.18.193:45395)
0808 18:21:41.418963 (+ 1384us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.418982 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.418998 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:21:41.419053 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.419056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.419167 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.419300 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:41.419612 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.420418 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.421997 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.422747 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.422842 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.423013 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.423064 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.423143 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":96,"threads_started":1}
I20260808 18:21:41.423851 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.418209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42003 (local address 127.0.0.1:36898)
0808 18:21:41.418951 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.418963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.419045 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.419879 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.419882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.419899 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:41.419966 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.419970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.421002 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.421005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.422040 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.422047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.422770 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.422774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.422885 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.422919 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.423026 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.423722 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":588,"thread_start_us":114,"threads_started":1}
I20260808 18:21:41.426019 23085 tablet_service.cc:1467] Processing CreateTablet for tablet aa3341aa9b1f4464b880c813d94c0e20 (DEFAULT_TABLE table=test-table [id=c919aaf9fc3d4a5fadaded4a3bc48f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:41.426090 23475 tablet_service.cc:1467] Processing CreateTablet for tablet aa3341aa9b1f4464b880c813d94c0e20 (DEFAULT_TABLE table=test-table [id=c919aaf9fc3d4a5fadaded4a3bc48f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:41.426837 23475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa3341aa9b1f4464b880c813d94c0e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:41.426841 23085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa3341aa9b1f4464b880c813d94c0e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:41.427748 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.417575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43025 (local address 127.0.0.1:38748)
0808 18:21:41.418613 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.418624 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.418706 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.419314 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.419316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.419342 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:41.419501 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.419507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.421585 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.421588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.422624 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.422635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.424122 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.424126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.427472 (+ 3346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.427495 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.427573 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.427633 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":34,"spinlock_wait_cycles":384,"thread_start_us":87,"threads_started":1}
I20260808 18:21:41.427891 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.418162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38748 (local address 127.27.18.194:43025)
0808 18:21:41.418510 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.418522 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.418834 (+ 312us) server_negotiation.cc:407] Connection header received
0808 18:21:41.418999 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.419007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.419184 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.419326 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:41.420536 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.421462 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.423124 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.424015 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.424134 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.424298 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.427619 (+ 3321us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.427710 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":160,"threads_started":1}
I20260808 18:21:41.430481 23275 tablet_service.cc:1467] Processing CreateTablet for tablet aa3341aa9b1f4464b880c813d94c0e20 (DEFAULT_TABLE table=test-table [id=c919aaf9fc3d4a5fadaded4a3bc48f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:41.431608 23275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa3341aa9b1f4464b880c813d94c0e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:41.432740 23606 tablet_bootstrap.cc:492] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Bootstrap starting.
I20260808 18:21:41.434676 23606 tablet_bootstrap.cc:654] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:41.435432 23606 log.cc:824] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:41.436132 23607 tablet_bootstrap.cc:492] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: Bootstrap starting.
I20260808 18:21:41.437378 23606 tablet_bootstrap.cc:492] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: No bootstrap required, opened a new log
I20260808 18:21:41.437564 23606 ts_tablet_manager.cc:1397] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:41.437724 23611 tablet_bootstrap.cc:492] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Bootstrap starting.
I20260808 18:21:41.438148 23607 tablet_bootstrap.cc:654] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:41.438887 23607 log.cc:824] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:41.439415 23611 tablet_bootstrap.cc:654] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:41.440066 23611 log.cc:824] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:41.440827 23607 tablet_bootstrap.cc:492] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: No bootstrap required, opened a new log
I20260808 18:21:41.440999 23607 ts_tablet_manager.cc:1397] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:41.441854 23611 tablet_bootstrap.cc:492] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: No bootstrap required, opened a new log
I20260808 18:21:41.441690 23606 raft_consensus.cc:348] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.441944 23606 raft_consensus.cc:374] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:41.442026 23606 raft_consensus.cc:729] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1cc5e2fcb441d08d2b355283eb653c, State: Initialized, Role: FOLLOWER
I20260808 18:21:41.442067 23611 ts_tablet_manager.cc:1397] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:41.442438 23606 consensus_queue.cc:260] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.443017 23580 heartbeater.cc:503] Master 127.27.18.254:36489 was elected leader, sending a full tablet report...
I20260808 18:21:41.443320 23606 ts_tablet_manager.cc:1428] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:21:41.444954 23607 raft_consensus.cc:348] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.445217 23607 raft_consensus.cc:374] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:41.445282 23607 raft_consensus.cc:729] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05a23ae59da48818c5296f95690e1a4, State: Initialized, Role: FOLLOWER
I20260808 18:21:41.445582 23611 raft_consensus.cc:348] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.445783 23611 raft_consensus.cc:374] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:41.445706 23607 consensus_queue.cc:260] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.445875 23611 raft_consensus.cc:729] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb68f0168d7c4d7698f225258a758ed4, State: Initialized, Role: FOLLOWER
I20260808 18:21:41.446290 23611 consensus_queue.cc:260] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.446359 23179 heartbeater.cc:503] Master 127.27.18.254:36489 was elected leader, sending a full tablet report...
I20260808 18:21:41.446676 23607 ts_tablet_manager.cc:1428] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 18:21:41.447141 23611 ts_tablet_manager.cc:1428] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:21:41.447495 23378 heartbeater.cc:503] Master 127.27.18.254:36489 was elected leader, sending a full tablet report...
W20260808 18:21:41.459227 23379 tablet.cc:2369] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:21:41.538254 23180 tablet.cc:2369] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:41.539081 23616 raft_consensus.cc:482] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:41.539242 23616 raft_consensus.cc:504] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.540230 23616 leader_election.cc:288] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb68f0168d7c4d7698f225258a758ed4 (127.27.18.194:43025), da1cc5e2fcb441d08d2b355283eb653c (127.27.18.195:42003)
I20260808 18:21:41.543602 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.540466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:33367 (local address 127.27.18.194:43025)
0808 18:21:41.540691 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.540696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.540710 (+ 14us) server_negotiation.cc:407] Connection header received
0808 18:21:41.540833 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.540838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.540890 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.540981 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:41.541230 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.541692 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.542668 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.543033 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.543154 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.543202 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.543398 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.543493 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260808 18:21:41.543603 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.540409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:43025 (local address 127.27.18.193:33367)
0808 18:21:41.540594 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.540609 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.540714 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.540993 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.540996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.541012 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:41.541122 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.541127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.541816 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.541819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.542553 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.542562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.543148 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.543151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.543241 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.543266 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.543358 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.543472 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":21}
I20260808 18:21:41.543736 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.540513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:53561 (local address 127.27.18.195:42003)
0808 18:21:41.540642 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.540646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.540901 (+ 255us) server_negotiation.cc:407] Connection header received
0808 18:21:41.541050 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.541056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.541092 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.541167 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:41.541419 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.541858 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.542816 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.543164 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.543284 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.543327 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.543557 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.543623 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:21:41.543756 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.540462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42003 (local address 127.27.18.193:53561)
0808 18:21:41.540816 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.540827 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:41.540950 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:41.541178 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:41.541181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:41.541195 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:41.541314 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.541334 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.541961 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.541964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.542699 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:41.542707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.543284 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.543288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:41.543411 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.543449 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:21:41.543542 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 18:21:41.543609 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":28,"thread_start_us":89,"threads_started":1}
I20260808 18:21:41.544267 23307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa3341aa9b1f4464b880c813d94c0e20" candidate_uuid: "c05a23ae59da48818c5296f95690e1a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb68f0168d7c4d7698f225258a758ed4" is_pre_election: true
I20260808 18:21:41.544281 23507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa3341aa9b1f4464b880c813d94c0e20" candidate_uuid: "c05a23ae59da48818c5296f95690e1a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1cc5e2fcb441d08d2b355283eb653c" is_pre_election: true
I20260808 18:21:41.544554 23507 raft_consensus.cc:2454] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c05a23ae59da48818c5296f95690e1a4 in term 0.
I20260808 18:21:41.544552 23307 raft_consensus.cc:2454] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c05a23ae59da48818c5296f95690e1a4 in term 0.
I20260808 18:21:41.545018 23022 leader_election.cc:302] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c05a23ae59da48818c5296f95690e1a4, da1cc5e2fcb441d08d2b355283eb653c; no voters:
I20260808 18:21:41.545248 23616 raft_consensus.cc:2781] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:41.545331 23616 raft_consensus.cc:482] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:41.545374 23616 raft_consensus.cc:3037] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:41.546286 23616 raft_consensus.cc:504] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.546576 23616 leader_election.cc:288] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [CANDIDATE]: Term 1 election: Requested vote from peers eb68f0168d7c4d7698f225258a758ed4 (127.27.18.194:43025), da1cc5e2fcb441d08d2b355283eb653c (127.27.18.195:42003)
I20260808 18:21:41.546885 23307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa3341aa9b1f4464b880c813d94c0e20" candidate_uuid: "c05a23ae59da48818c5296f95690e1a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb68f0168d7c4d7698f225258a758ed4"
I20260808 18:21:41.546978 23507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa3341aa9b1f4464b880c813d94c0e20" candidate_uuid: "c05a23ae59da48818c5296f95690e1a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1cc5e2fcb441d08d2b355283eb653c"
I20260808 18:21:41.546989 23307 raft_consensus.cc:3037] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:41.547122 23507 raft_consensus.cc:3037] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:41.548044 23307 raft_consensus.cc:2454] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c05a23ae59da48818c5296f95690e1a4 in term 1.
I20260808 18:21:41.548213 23507 raft_consensus.cc:2454] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c05a23ae59da48818c5296f95690e1a4 in term 1.
I20260808 18:21:41.548353 23021 leader_election.cc:302] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c05a23ae59da48818c5296f95690e1a4, eb68f0168d7c4d7698f225258a758ed4; no voters:
I20260808 18:21:41.548583 23616 raft_consensus.cc:2781] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:41.548806 23616 raft_consensus.cc:686] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [term 1 LEADER]: Becoming Leader. State: Replica: c05a23ae59da48818c5296f95690e1a4, State: Running, Role: LEADER
I20260808 18:21:41.549099 23616 consensus_queue.cc:237] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } }
I20260808 18:21:41.551898 22919 catalog_manager.cc:5495] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c05a23ae59da48818c5296f95690e1a4 (127.27.18.193). New cstate: current_term: 1 leader_uuid: "c05a23ae59da48818c5296f95690e1a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c05a23ae59da48818c5296f95690e1a4" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 } health_report { overall_health: UNKNOWN } } }
I20260808 18:21:41.574050 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.571005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58488 (local address 127.27.18.193:45395)
0808 18:21:41.571128 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:41.571131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:41.571182 (+ 51us) server_negotiation.cc:407] Connection header received
0808 18:21:41.571316 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:41.571318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:41.571411 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:41.571508 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:41.571695 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.572106 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.572868 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:41.573077 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:41.573143 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:41.573399 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:41.573765 (+ 366us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:41.573858 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:41.573910 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 18:21:41.573949 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:21:41.579089 23307 raft_consensus.cc:1261] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 1 FOLLOWER]: Refusing update from remote peer c05a23ae59da48818c5296f95690e1a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:41.579089 23507 raft_consensus.cc:1261] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 1 FOLLOWER]: Refusing update from remote peer c05a23ae59da48818c5296f95690e1a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:41.579826 23616 consensus_queue.cc:1035] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb68f0168d7c4d7698f225258a758ed4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:41.579993 23621 consensus_queue.cc:1035] T aa3341aa9b1f4464b880c813d94c0e20 P c05a23ae59da48818c5296f95690e1a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1cc5e2fcb441d08d2b355283eb653c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:41.587015 23629 mvcc.cc:205] Tried to move back new op lower bound from 7316329683262341120 to 7316329683147632640. Current Snapshot: MvccSnapshot[applied={T|T < 7316329683262341120}]
I20260808 18:21:41.594347 22918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:41.594499 22918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:41.597208 22918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81bee03854a6441b97a6598de17e7a87: Sending DeleteTablet for 3 replicas of tablet aa3341aa9b1f4464b880c813d94c0e20
I20260808 18:21:41.597913 23085 tablet_service.cc:1514] Processing DeleteTablet for tablet aa3341aa9b1f4464b880c813d94c0e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:41 UTC) from {username='slave'} at 127.0.0.1:58486
I20260808 18:21:41.597968 23475 tablet_service.cc:1514] Processing DeleteTablet for tablet aa3341aa9b1f4464b880c813d94c0e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:41 UTC) from {username='slave'} at 127.0.0.1:36898
I20260808 18:21:41.598469 23275 tablet_service.cc:1514] Processing DeleteTablet for tablet aa3341aa9b1f4464b880c813d94c0e20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:41 UTC) from {username='slave'} at 127.0.0.1:38748
I20260808 18:21:41.598850 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 23000
I20260808 18:21:41.610035 23643 tablet_replica.cc:330] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: stopping tablet replica
I20260808 18:21:41.610365 23643 raft_consensus.cc:2229] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:41.610652 23643 raft_consensus.cc:2258] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:41.611063 23644 tablet_replica.cc:330] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: stopping tablet replica
I20260808 18:21:41.611331 23644 raft_consensus.cc:2229] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:41.611624 23644 raft_consensus.cc:2258] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:41.612212 23643 ts_tablet_manager.cc:1905] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:41.612922 23644 ts_tablet_manager.cc:1905] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:41.615091 23643 ts_tablet_manager.cc:1918] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:41.615187 23643 log.cc:1196] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/aa3341aa9b1f4464b880c813d94c0e20
I20260808 18:21:41.615537 23643 ts_tablet_manager.cc:1939] T aa3341aa9b1f4464b880c813d94c0e20 P da1cc5e2fcb441d08d2b355283eb653c: Deleting consensus metadata
I20260808 18:21:41.615757 23644 ts_tablet_manager.cc:1918] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:41.615862 23644 log.cc:1196] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/aa3341aa9b1f4464b880c813d94c0e20
I20260808 18:21:41.616171 23644 ts_tablet_manager.cc:1939] T aa3341aa9b1f4464b880c813d94c0e20 P eb68f0168d7c4d7698f225258a758ed4: Deleting consensus metadata
I20260808 18:21:41.617134 22904 catalog_manager.cc:4841] TS da1cc5e2fcb441d08d2b355283eb653c (127.27.18.195:42003): tablet aa3341aa9b1f4464b880c813d94c0e20 (table test-table [id=c919aaf9fc3d4a5fadaded4a3bc48f87]) successfully deleted
W20260808 18:21:41.617266 22901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:45395 (error 108)
I20260808 18:21:41.617252 22902 catalog_manager.cc:4841] TS eb68f0168d7c4d7698f225258a758ed4 (127.27.18.194:43025): tablet aa3341aa9b1f4464b880c813d94c0e20 (table test-table [id=c919aaf9fc3d4a5fadaded4a3bc48f87]) successfully deleted
I20260808 18:21:41.617422 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 23182
I20260808 18:21:41.620011 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:41.619768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:45395 (local address 127.0.0.1:58500)
0808 18:21:41.619885 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:41.619945 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:45395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:21:41.620850 22901 catalog_manager.cc:4584] TS c05a23ae59da48818c5296f95690e1a4 (127.27.18.193:45395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa3341aa9b1f4464b880c813d94c0e20: Network error: Client connection negotiation failed: client connection to 127.27.18.193:45395: connect: Connection refused (error 111)
I20260808 18:21:41.626173 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 23382
I20260808 18:21:41.638324 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 22878
2026-08-08T18:21:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1053 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:41 dist-test-slave-x4qh krb5kdc[23650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:41 dist-test-slave-x4qh krb5kdc[23650](info): set up 2 sockets
Aug 08 18:21:41 dist-test-slave-x4qh krb5kdc[23650](info): commencing operation
krb5kdc: starting...
W20260808 18:21:42.806710 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:42 dist-test-slave-x4qh krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:21:42.920331 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:45401
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:45401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:43.023272 23668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:43.023566 23668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:43.023664 23668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:43.027484 23668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:43.027529 23668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:43.027549 23668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:43.027565 23668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:43.027580 23668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:43.032354 23668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:45401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:45401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:43.033226 23668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:43.034132 23668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:43.042155 23674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.042191 23676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.042234 23673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:43.042306 23668 server_base.cc:1034] running on GCE node
I20260808 18:21:43.043027 23668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:43.044109 23668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:43.045305 23668 hybrid_clock.cc:648] HybridClock initialized: now 1786213303045280 us; error 42 us; skew 500 ppm
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:43.053434 23668 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:21:43.054270 23668 webserver.cc:460] Webserver started at http://127.27.18.254:43853/ using document root <none> and password file <none>
I20260808 18:21:43.054808 23668 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:43.054869 23668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:43.055073 23668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:43.057070 23668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "172ffa4c53b540718589f4b881db5066"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "fc1a29ff89818a44e3bd0ec0a02bb6d2"
server_key_iv: "b37b05d4db7477e6fccad44cc1b7fa54"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.057623 23668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "172ffa4c53b540718589f4b881db5066"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "fc1a29ff89818a44e3bd0ec0a02bb6d2"
server_key_iv: "b37b05d4db7477e6fccad44cc1b7fa54"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.061082 23668 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:21:43.063177 23683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.064373 23668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:43.064491 23668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "172ffa4c53b540718589f4b881db5066"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "fc1a29ff89818a44e3bd0ec0a02bb6d2"
server_key_iv: "b37b05d4db7477e6fccad44cc1b7fa54"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.064585 23668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:43.075870 23668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:43.078619 23668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:43.078752 23668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:43.085666 23668 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:45401
I20260808 18:21:43.085696 23760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:45401 every 8 connection(s)
I20260808 18:21:43.086500 23668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:43.088903 23762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:43.093729 23762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: Bootstrap starting.
I20260808 18:21:43.095590 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 23668
I20260808 18:21:43.095724 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:43.096007 23762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:43.096011 27723 external_mini_cluster.cc:1428] Setting key d63003d5a3aba06ec99724ea8a019cf8
I20260808 18:21:43.096794 23762 log.cc:824] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:43.098662 23762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: No bootstrap required, opened a new log
I20260808 18:21:43.101816 23762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172ffa4c53b540718589f4b881db5066" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45401 } }
I20260808 18:21:43.102037 23762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:43.102101 23762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 172ffa4c53b540718589f4b881db5066, State: Initialized, Role: FOLLOWER
I20260808 18:21:43.102540 23762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172ffa4c53b540718589f4b881db5066" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45401 } }
I20260808 18:21:43.102653 23762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:43.102725 23762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:43.102836 23762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:43.103761 23762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172ffa4c53b540718589f4b881db5066" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45401 } }
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:43.104152 23762 leader_election.cc:302] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 172ffa4c53b540718589f4b881db5066; no voters:
I20260808 18:21:43.104530 23762 leader_election.cc:288] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:43.104635 23770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:43.104851 23770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [term 1 LEADER]: Becoming Leader. State: Replica: 172ffa4c53b540718589f4b881db5066, State: Running, Role: LEADER
I20260808 18:21:43.105489 23770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172ffa4c53b540718589f4b881db5066" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45401 } }
I20260808 18:21:43.105751 23762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:43.107293 23772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 172ffa4c53b540718589f4b881db5066. Latest consensus state: current_term: 1 leader_uuid: "172ffa4c53b540718589f4b881db5066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172ffa4c53b540718589f4b881db5066" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45401 } } }
I20260808 18:21:43.107342 23771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "172ffa4c53b540718589f4b881db5066" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172ffa4c53b540718589f4b881db5066" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45401 } } }
I20260808 18:21:43.107542 23771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:43.107816 23772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:43.110208 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.097021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.27.18.254:45401)
0808 18:21:43.097508 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.097527 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.097569 (+ 42us) server_negotiation.cc:407] Connection header received
0808 18:21:43.098168 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.098183 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.098472 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.098764 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:43.099584 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.100471 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.101282 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.101728 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.101898 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.104631 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:43.104638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:43.104656 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:43.104693 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:43.106662 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.107236 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.107244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.107249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.107410 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.107755 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.107761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.107763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.108245 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:43.108452 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.109027 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.109303 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":140,"threads_started":1}
W20260808 18:21:43.110471 23796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:21:43.110549 23796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:21:43.110630 23794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:43.111500 23794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:43.116178 23794 catalog_manager.cc:1347] Generated new cluster ID: 14fc008e16ff4fd09d0220126be59605
I20260808 18:21:43.116246 23794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:43.154008 23794 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:43.154910 23794 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:43.163203 23794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: Generated new TSK 0
I20260808 18:21:43.163800 23794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:21:43.194509 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:45401
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:43.296629 23806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:43.296825 23806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:43.296861 23806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:43.300724 23806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:43.300767 23806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:43.300819 23806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:43.305671 23806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:45401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:43.306510 23806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:43.307492 23806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:43.316250 23818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.316247 23815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.316247 23816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:43.316550 23806 server_base.cc:1034] running on GCE node
I20260808 18:21:43.316996 23806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:43.317579 23806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:43.318756 23806 hybrid_clock.cc:648] HybridClock initialized: now 1786213303318742 us; error 24 us; skew 500 ppm
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:43.321461 23806 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:21:43.322247 23806 webserver.cc:460] Webserver started at http://127.27.18.193:32831/ using document root <none> and password file <none>
I20260808 18:21:43.322721 23806 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:43.322775 23806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:43.322929 23806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:43.324769 23806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "6d412dd0e2e24b32accfb26f0291ad6a"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "e7ed8bf3edb611d4dc963cccca1b6f4a"
server_key_iv: "6c17b1e38079ab33927ab9d2df63e6fb"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.325265 23806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "6d412dd0e2e24b32accfb26f0291ad6a"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "e7ed8bf3edb611d4dc963cccca1b6f4a"
server_key_iv: "6c17b1e38079ab33927ab9d2df63e6fb"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.328588 23806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 18:21:43.330809 23826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.331983 23806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:43.332074 23806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "6d412dd0e2e24b32accfb26f0291ad6a"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "e7ed8bf3edb611d4dc963cccca1b6f4a"
server_key_iv: "6c17b1e38079ab33927ab9d2df63e6fb"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.332151 23806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:43.343888 23806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:43.346237 23806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:43.346383 23806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:43.346827 23806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:43.347594 23806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:43.347638 23806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.347671 23806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:43.347692 23806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.356140 23806 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:38785
I20260808 18:21:43.356164 23985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:38785 every 8 connection(s)
I20260808 18:21:43.356876 23806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:43.359903 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 23806
I20260808 18:21:43.360040 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:21:43.360353 27723 external_mini_cluster.cc:1428] Setting key cdc7a1d9c79c3bfef6bc16e6e0314560
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:43.370963 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.358404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:37421 (local address 127.27.18.254:45401)
0808 18:21:43.358548 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.358553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.359225 (+ 672us) server_negotiation.cc:407] Connection header received
0808 18:21:43.360769 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.360773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.360841 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.360939 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:43.362251 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.362767 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.364442 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.364837 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.364968 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.367445 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:43.367452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:43.367457 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:43.367488 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:43.368965 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.369524 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.369528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.369530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.369610 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.369943 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.369947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.369949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.370114 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:43.370221 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.370660 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.370782 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:21:43.371734 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.358714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45401 (local address 127.27.18.193:37421)
0808 18:21:43.359095 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.359130 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.360552 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.361119 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.361127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.361504 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:43.362097 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.362118 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.362898 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.362905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.364304 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.364314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.364986 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.364989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.365112 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.365636 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:43.365658 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:43.367245 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:43.369080 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:43.369086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:43.369098 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:43.369422 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:43.369703 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:43.369707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:43.369709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:43.369845 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:43.370221 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:43.370228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:43.370446 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.370618 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.370907 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":99,"threads_started":1}
I20260808 18:21:43.372877 23986 heartbeater.cc:347] Connected to a master server at 127.27.18.254:45401
I20260808 18:21:43.373095 23986 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:43.373533 23986 heartbeater.cc:511] Master 127.27.18.254:45401 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:21:43.375059 23710 ts_manager.cc:194] Registered new tserver with Master: 6d412dd0e2e24b32accfb26f0291ad6a (127.27.18.193:38785)
I20260808 18:21:43.376518 23710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:37421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:21:43.388479 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:45401
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:43.489251 23993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:43.489444 23993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:43.489480 23993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:43.492967 23993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:43.493006 23993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:43.493054 23993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:43.497548 23993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:45401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:43.498282 23993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:43.499045 23993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:43.508147 24000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.508142 24004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.508338 23999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:43.511027 23993 server_base.cc:1034] running on GCE node
I20260808 18:21:43.511437 23993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:43.512140 23993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:43.513293 23993 hybrid_clock.cc:648] HybridClock initialized: now 1786213303513270 us; error 37 us; skew 500 ppm
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:43.516520 23993 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:21:43.517309 23993 webserver.cc:460] Webserver started at http://127.27.18.194:42723/ using document root <none> and password file <none>
I20260808 18:21:43.517812 23993 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:43.517900 23993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:43.518092 23993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:43.519891 23993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "de5deecab17343a89c403fbb9dc9ad48"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "c4a7f2fd90b9dc283cb9930cad07f126"
server_key_iv: "057f97b7b9ebcc2199af408d65c3c4e2"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.520385 23993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "de5deecab17343a89c403fbb9dc9ad48"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "c4a7f2fd90b9dc283cb9930cad07f126"
server_key_iv: "057f97b7b9ebcc2199af408d65c3c4e2"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.523833 23993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:21:43.525954 24014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.526942 23993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:43.527069 23993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "de5deecab17343a89c403fbb9dc9ad48"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "c4a7f2fd90b9dc283cb9930cad07f126"
server_key_iv: "057f97b7b9ebcc2199af408d65c3c4e2"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.527169 23993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:43.571930 23993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:43.574564 23993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:43.574750 23993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:43.575277 23993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:43.576037 23993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:43.576102 23993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.576166 23993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:43.576216 23993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.584362 23993 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:36061
I20260808 18:21:43.584368 24181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:36061 every 8 connection(s)
I20260808 18:21:43.585183 23993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:43.594735 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 23993
I20260808 18:21:43.594910 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:21:43.595256 27723 external_mini_cluster.cc:1428] Setting key ee8dd8d7ba93f6021693b926872ddb0c
I20260808 18:21:43.597359 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.586860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:57039 (local address 127.27.18.254:45401)
0808 18:21:43.586992 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.586995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.587752 (+ 757us) server_negotiation.cc:407] Connection header received
0808 18:21:43.588856 (+ 1104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.588859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.588903 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.588972 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:43.589930 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.590399 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.591365 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.591583 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.591648 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.593778 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:43.593781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:43.593783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:43.593806 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:43.595258 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.595823 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.595828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.595829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.595909 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.596253 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.596255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.596256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.596450 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:43.596531 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.597023 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.597138 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:21:43.597920 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.587173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45401 (local address 127.27.18.194:57039)
0808 18:21:43.587618 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.587656 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.588678 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.589118 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.589123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.589413 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:43.589815 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.589826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.590496 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.590498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.591269 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.591277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.591690 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.591692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.591817 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.592306 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:43.592321 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:43.593635 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:43.595415 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:43.595426 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:43.595435 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:43.595700 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:43.596006 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:43.596009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:43.596010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:43.596166 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:43.596556 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:43.596561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:43.596794 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.596925 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.597141 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":167,"threads_started":1}
I20260808 18:21:43.598997 24182 heartbeater.cc:347] Connected to a master server at 127.27.18.254:45401
I20260808 18:21:43.599211 24182 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:43.599669 24182 heartbeater.cc:511] Master 127.27.18.254:45401 requested a full tablet report, sending...
I20260808 18:21:43.600620 23710 ts_manager.cc:194] Registered new tserver with Master: de5deecab17343a89c403fbb9dc9ad48 (127.27.18.194:36061)
I20260808 18:21:43.601187 23710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:57039
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:21:43.622097 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:45401
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:43.726547 24190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:43.726758 24190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:43.726794 24190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:43.730469 24190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:43.730513 24190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:43.730604 24190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:43.735436 24190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:45401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:43.736353 24190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:43.737263 24190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:43.745889 24199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.745900 24203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:43.745885 24200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:43.746220 24190 server_base.cc:1034] running on GCE node
I20260808 18:21:43.746609 24190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:43.747155 24190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:43.748337 24190 hybrid_clock.cc:648] HybridClock initialized: now 1786213303748302 us; error 46 us; skew 500 ppm
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:43.750690 24190 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:21:43.751518 24190 webserver.cc:460] Webserver started at http://127.27.18.195:42341/ using document root <none> and password file <none>
I20260808 18:21:43.752017 24190 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:43.752104 24190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:43.752318 24190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:43.754083 24190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "71d26ec0d02342299d7bd3f4fc69be26"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "ba455268aa66eaa531cf85259dfc8801"
server_key_iv: "8aa3686d7a1449ab7d697924dadfab5c"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.754639 24190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "71d26ec0d02342299d7bd3f4fc69be26"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "ba455268aa66eaa531cf85259dfc8801"
server_key_iv: "8aa3686d7a1449ab7d697924dadfab5c"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.758044 24190 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:21:43.760205 24214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.761381 24190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:43.761531 24190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "71d26ec0d02342299d7bd3f4fc69be26"
format_stamp: "Formatted at 2026-08-08 18:21:43 on dist-test-slave-x4qh"
server_key: "ba455268aa66eaa531cf85259dfc8801"
server_key_iv: "8aa3686d7a1449ab7d697924dadfab5c"
server_key_version: "encryptionkey@0"
I20260808 18:21:43.761641 24190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:43.785905 24190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:43.788465 24190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:43.788640 24190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:43.789250 24190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:43.790205 24190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:43.790259 24190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.790343 24190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:43.790376 24190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:43.798908 24190 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:35701
I20260808 18:21:43.798930 24375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:35701 every 8 connection(s)
I20260808 18:21:43.799693 24190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:43.807641 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 24190
I20260808 18:21:43.807785 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
Aug 08 18:21:43 dist-test-slave-x4qh krb5kdc[23650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213303, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:43.808195 27723 external_mini_cluster.cc:1428] Setting key 906f7842804cc08f1be5af0fb7d6a22b
I20260808 18:21:43.812177 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.801304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:38799 (local address 127.27.18.254:45401)
0808 18:21:43.801447 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.801451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.802179 (+ 728us) server_negotiation.cc:407] Connection header received
0808 18:21:43.803309 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.803311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.803356 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.803490 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:43.804477 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.804973 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.805965 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.806212 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.806283 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.808739 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:43.808743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:43.808746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:43.808771 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:43.810440 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.810901 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.810904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.810905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.810988 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.811262 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.811264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.811265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.811490 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:43.811561 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.811969 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.812056 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:21:43.812902 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.801629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45401 (local address 127.27.18.195:38799)
0808 18:21:43.802042 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.802081 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.803131 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.803634 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.803641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.803936 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:43.804357 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.804370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.805074 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.805077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.805869 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.805877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.806338 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.806341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.806464 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.806892 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:43.806906 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:43.808580 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:43.810574 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:43.810579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:43.810589 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:43.810822 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:43.811066 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:43.811068 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:43.811069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:43.811186 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:43.811576 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:43.811581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:43.811765 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.811921 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.812131 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":109,"threads_started":1}
I20260808 18:21:43.813885 24376 heartbeater.cc:347] Connected to a master server at 127.27.18.254:45401
I20260808 18:21:43.814131 24376 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:43.814585 24376 heartbeater.cc:511] Master 127.27.18.254:45401 requested a full tablet report, sending...
I20260808 18:21:43.815527 23710 ts_manager.cc:194] Registered new tserver with Master: 71d26ec0d02342299d7bd3f4fc69be26 (127.27.18.195:35701)
I20260808 18:21:43.816020 23710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:38799
I20260808 18:21:43.821898 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:43.830852 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.823878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33776 (local address 127.27.18.254:45401)
0808 18:21:43.823994 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.823997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.824173 (+ 176us) server_negotiation.cc:407] Connection header received
0808 18:21:43.824249 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.824251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.824285 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.824364 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:43.824999 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.825477 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.826250 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.826446 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.826516 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.828014 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:43.828017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:43.828019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:43.828039 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:43.829186 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.829629 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.829632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.829633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.829723 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:43.830003 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:43.830006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:43.830007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:43.830182 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:43.830287 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.830632 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.830717 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 18:21:43.833534 23710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:43.836594 23710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:43.850306 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.844428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:36061 (local address 127.0.0.1:48350)
0808 18:21:43.845068 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.845113 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.845227 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.846318 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.846321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.846338 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:43.846461 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.846467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.847932 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.847935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.848899 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.848909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.849958 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.849960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.850030 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.850056 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.850131 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.850201 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":57,"thread_start_us":56,"threads_started":1}
I20260808 18:21:43.850378 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.844568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.27.18.194:36061)
0808 18:21:43.845895 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.845906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.845925 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:21:43.845992 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.845996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.846169 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.846614 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:43.846762 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.847806 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.849017 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.849841 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.849945 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.850073 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.850158 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.850248 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1242,"thread_start_us":131,"threads_started":1}
I20260808 18:21:43.851037 24395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.844525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35701 (local address 127.0.0.1:45634)
0808 18:21:43.845313 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.845326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.845394 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.846291 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.846297 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.846318 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:43.846494 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.846501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.847883 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.847886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.849027 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.849033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.850686 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.850691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.850771 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.850792 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.850871 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.850930 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":95,"thread_start_us":103,"threads_started":1}
I20260808 18:21:43.851430 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.844811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38785 (local address 127.0.0.1:51004)
0808 18:21:43.845580 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.845591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.845658 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.846443 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.846445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.846462 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:43.846575 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.846581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.847602 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.847605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.849011 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.849022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.850907 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.850910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.851128 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.851156 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.851224 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.851289 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":152,"thread_start_us":104,"threads_started":1}
I20260808 18:21:43.851698 24397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.845595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.27.18.193:38785)
0808 18:21:43.845882 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.845896 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.845920 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:21:43.845992 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.845996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.846169 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.846318 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:43.846695 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.847480 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.850132 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.850779 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.850895 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.851024 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.851464 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.851567 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
I20260808 18:21:43.851984 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.845164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45634 (local address 127.27.18.195:35701)
0808 18:21:43.845939 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.845947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.845964 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:21:43.846020 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.846023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.846148 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.846298 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:43.846613 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.847730 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.849144 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.849854 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.849945 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.850073 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.851443 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.851861 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":150,"threads_started":1}
I20260808 18:21:43.853480 24090 tablet_service.cc:1467] Processing CreateTablet for tablet 26adc32d903a44808e7b67feeef75cd2 (DEFAULT_TABLE table=test-table [id=a9369413cb414ae489bd75749e7903ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:43.854238 24282 tablet_service.cc:1467] Processing CreateTablet for tablet 26adc32d903a44808e7b67feeef75cd2 (DEFAULT_TABLE table=test-table [id=a9369413cb414ae489bd75749e7903ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:43.854626 24090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26adc32d903a44808e7b67feeef75cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:43.854259 23890 tablet_service.cc:1467] Processing CreateTablet for tablet 26adc32d903a44808e7b67feeef75cd2 (DEFAULT_TABLE table=test-table [id=a9369413cb414ae489bd75749e7903ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:43.854990 24282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26adc32d903a44808e7b67feeef75cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:43.855196 23890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26adc32d903a44808e7b67feeef75cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:43.860724 24402 tablet_bootstrap.cc:492] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Bootstrap starting.
I20260808 18:21:43.862200 24405 tablet_bootstrap.cc:492] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a: Bootstrap starting.
I20260808 18:21:43.862200 24404 tablet_bootstrap.cc:492] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Bootstrap starting.
I20260808 18:21:43.863706 24402 tablet_bootstrap.cc:654] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:43.864334 24405 tablet_bootstrap.cc:654] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:43.864336 24404 tablet_bootstrap.cc:654] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:43.864933 24402 log.cc:824] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:43.865648 24404 log.cc:824] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:43.865648 24405 log.cc:824] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:43.867237 24402 tablet_bootstrap.cc:492] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: No bootstrap required, opened a new log
I20260808 18:21:43.867460 24402 ts_tablet_manager.cc:1397] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:21:43.867703 24405 tablet_bootstrap.cc:492] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a: No bootstrap required, opened a new log
I20260808 18:21:43.867897 24405 ts_tablet_manager.cc:1397] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:21:43.870414 24402 raft_consensus.cc:348] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.870604 24402 raft_consensus.cc:374] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:43.870636 24402 raft_consensus.cc:729] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d26ec0d02342299d7bd3f4fc69be26, State: Initialized, Role: FOLLOWER
I20260808 18:21:43.870545 24405 raft_consensus.cc:348] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.870844 24405 raft_consensus.cc:374] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:43.870903 24405 raft_consensus.cc:729] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d412dd0e2e24b32accfb26f0291ad6a, State: Initialized, Role: FOLLOWER
I20260808 18:21:43.871132 24402 consensus_queue.cc:260] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.871234 24405 consensus_queue.cc:260] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.871749 24376 heartbeater.cc:503] Master 127.27.18.254:45401 was elected leader, sending a full tablet report...
I20260808 18:21:43.872090 24402 ts_tablet_manager.cc:1428] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260808 18:21:43.872351 24405 ts_tablet_manager.cc:1428] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 18:21:43.872648 23986 heartbeater.cc:503] Master 127.27.18.254:45401 was elected leader, sending a full tablet report...
I20260808 18:21:43.872861 24404 tablet_bootstrap.cc:492] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: No bootstrap required, opened a new log
I20260808 18:21:43.873040 24404 ts_tablet_manager.cc:1397] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 18:21:43.876691 24404 raft_consensus.cc:348] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.876876 24404 raft_consensus.cc:374] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:43.876909 24404 raft_consensus.cc:729] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de5deecab17343a89c403fbb9dc9ad48, State: Initialized, Role: FOLLOWER
I20260808 18:21:43.877187 24404 consensus_queue.cc:260] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.877722 24182 heartbeater.cc:503] Master 127.27.18.254:45401 was elected leader, sending a full tablet report...
I20260808 18:21:43.877801 24404 ts_tablet_manager.cc:1428] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:21:43.982877 24415 raft_consensus.cc:482] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:43.983059 24415 raft_consensus.cc:504] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.984180 24415 leader_election.cc:288] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de5deecab17343a89c403fbb9dc9ad48 (127.27.18.194:36061), 71d26ec0d02342299d7bd3f4fc69be26 (127.27.18.195:35701)
I20260808 18:21:43.987903 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.984381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:36061 (local address 127.27.18.193:58041)
0808 18:21:43.984718 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.984732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.984832 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.985071 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.985074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.985089 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:43.985231 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.985240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.986003 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.986009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.986852 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.986861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.987486 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.987489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.987578 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.987610 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.987681 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.987754 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":23,"thread_start_us":102,"threads_started":1}
I20260808 18:21:43.987954 24419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.984466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:35701 (local address 127.27.18.193:36417)
0808 18:21:43.984806 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:43.984822 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:43.984927 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:43.985197 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:43.985199 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:43.985214 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:43.985348 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.985355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.986065 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.986068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.986858 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:43.986866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.987541 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.987545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:43.987645 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.987682 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 18:21:43.987761 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 18:21:43.987827 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260808 18:21:43.987937 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.984467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:58041 (local address 127.27.18.194:36061)
0808 18:21:43.984609 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.984616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.984837 (+ 221us) server_negotiation.cc:407] Connection header received
0808 18:21:43.984906 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.984910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.984965 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.985049 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:43.985364 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.985885 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.986980 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.987361 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.987502 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.987557 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.987724 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.987804 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:21:43.987998 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:43.984566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:36417 (local address 127.27.18.195:35701)
0808 18:21:43.984687 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:43.984691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:43.984927 (+ 236us) server_negotiation.cc:407] Connection header received
0808 18:21:43.985003 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:43.985005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:43.985078 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:43.985154 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:43.985464 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.985950 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.986986 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:43.987409 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:43.987526 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:43.987578 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:43.987806 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 18:21:43.987876 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:21:43.988499 24113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26adc32d903a44808e7b67feeef75cd2" candidate_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de5deecab17343a89c403fbb9dc9ad48" is_pre_election: true
I20260808 18:21:43.988524 24310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26adc32d903a44808e7b67feeef75cd2" candidate_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d26ec0d02342299d7bd3f4fc69be26" is_pre_election: true
I20260808 18:21:43.988786 24310 raft_consensus.cc:2454] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d412dd0e2e24b32accfb26f0291ad6a in term 0.
I20260808 18:21:43.988788 24113 raft_consensus.cc:2454] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d412dd0e2e24b32accfb26f0291ad6a in term 0.
I20260808 18:21:43.989215 23830 leader_election.cc:302] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d412dd0e2e24b32accfb26f0291ad6a, de5deecab17343a89c403fbb9dc9ad48; no voters:
I20260808 18:21:43.989491 24415 raft_consensus.cc:2781] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:43.989596 24415 raft_consensus.cc:482] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:43.989643 24415 raft_consensus.cc:3037] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:43.990621 24415 raft_consensus.cc:504] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.990909 24415 leader_election.cc:288] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [CANDIDATE]: Term 1 election: Requested vote from peers de5deecab17343a89c403fbb9dc9ad48 (127.27.18.194:36061), 71d26ec0d02342299d7bd3f4fc69be26 (127.27.18.195:35701)
I20260808 18:21:43.991209 24310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26adc32d903a44808e7b67feeef75cd2" candidate_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d26ec0d02342299d7bd3f4fc69be26"
I20260808 18:21:43.991228 24113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26adc32d903a44808e7b67feeef75cd2" candidate_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de5deecab17343a89c403fbb9dc9ad48"
I20260808 18:21:43.991333 24310 raft_consensus.cc:3037] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:43.991324 24113 raft_consensus.cc:3037] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:43.992550 24113 raft_consensus.cc:2454] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d412dd0e2e24b32accfb26f0291ad6a in term 1.
I20260808 18:21:43.992550 24310 raft_consensus.cc:2454] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d412dd0e2e24b32accfb26f0291ad6a in term 1.
I20260808 18:21:43.992847 23830 leader_election.cc:302] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d412dd0e2e24b32accfb26f0291ad6a, de5deecab17343a89c403fbb9dc9ad48; no voters:
I20260808 18:21:43.993013 24415 raft_consensus.cc:2781] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:43.993209 24415 raft_consensus.cc:686] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [term 1 LEADER]: Becoming Leader. State: Replica: 6d412dd0e2e24b32accfb26f0291ad6a, State: Running, Role: LEADER
I20260808 18:21:43.993535 24415 consensus_queue.cc:237] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } }
I20260808 18:21:43.996516 23709 catalog_manager.cc:5495] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d412dd0e2e24b32accfb26f0291ad6a (127.27.18.193). New cstate: current_term: 1 leader_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d412dd0e2e24b32accfb26f0291ad6a" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 38785 } health_report { overall_health: HEALTHY } } }
I20260808 18:21:44.036880 24397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.033585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.27.18.193:38785)
0808 18:21:44.033702 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.033708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.033719 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:21:44.033848 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.033850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.033898 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.033981 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:44.034187 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.034633 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.035527 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.035732 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.035835 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.036042 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:44.036432 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:44.036577 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.036712 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.036753 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:21:44.042303 24113 raft_consensus.cc:1261] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 1 FOLLOWER]: Refusing update from remote peer 6d412dd0e2e24b32accfb26f0291ad6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:44.042315 24310 raft_consensus.cc:1261] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 1 FOLLOWER]: Refusing update from remote peer 6d412dd0e2e24b32accfb26f0291ad6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:44.043020 24420 consensus_queue.cc:1035] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d26ec0d02342299d7bd3f4fc69be26" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:44.043152 24415 consensus_queue.cc:1035] T 26adc32d903a44808e7b67feeef75cd2 P 6d412dd0e2e24b32accfb26f0291ad6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "de5deecab17343a89c403fbb9dc9ad48" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 18:21:44.050518 24378 tablet.cc:2369] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:44.058537 23710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:44.058660 23710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:44.060673 23710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 172ffa4c53b540718589f4b881db5066: Sending DeleteTablet for 3 replicas of tablet 26adc32d903a44808e7b67feeef75cd2
I20260808 18:21:44.061352 24090 tablet_service.cc:1514] Processing DeleteTablet for tablet 26adc32d903a44808e7b67feeef75cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:44 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:48350
I20260808 18:21:44.061375 24282 tablet_service.cc:1514] Processing DeleteTablet for tablet 26adc32d903a44808e7b67feeef75cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:44 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:45634
I20260808 18:21:44.061959 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 23806
I20260808 18:21:44.069877 24439 tablet_replica.cc:330] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: stopping tablet replica
I20260808 18:21:44.070209 24439 raft_consensus.cc:2229] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:44.070467 24439 raft_consensus.cc:2258] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:44.071251 24436 tablet_replica.cc:330] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: stopping tablet replica
I20260808 18:21:44.071527 24436 raft_consensus.cc:2229] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:44.071780 24436 raft_consensus.cc:2258] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:44.072134 24439 ts_tablet_manager.cc:1905] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:44.072908 24436 ts_tablet_manager.cc:1905] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:44.075014 24436 ts_tablet_manager.cc:1918] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:44.075124 24436 log.cc:1196] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/26adc32d903a44808e7b67feeef75cd2
I20260808 18:21:44.075449 24436 ts_tablet_manager.cc:1939] T 26adc32d903a44808e7b67feeef75cd2 P de5deecab17343a89c403fbb9dc9ad48: Deleting consensus metadata
I20260808 18:21:44.075657 24439 ts_tablet_manager.cc:1918] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:44.075765 24439 log.cc:1196] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/26adc32d903a44808e7b67feeef75cd2
I20260808 18:21:44.076092 24439 ts_tablet_manager.cc:1939] T 26adc32d903a44808e7b67feeef75cd2 P 71d26ec0d02342299d7bd3f4fc69be26: Deleting consensus metadata
I20260808 18:21:44.076617 23689 catalog_manager.cc:4841] TS de5deecab17343a89c403fbb9dc9ad48 (127.27.18.194:36061): tablet 26adc32d903a44808e7b67feeef75cd2 (table test-table [id=a9369413cb414ae489bd75749e7903ee]) successfully deleted
I20260808 18:21:44.077281 23688 catalog_manager.cc:4841] TS 71d26ec0d02342299d7bd3f4fc69be26 (127.27.18.195:35701): tablet 26adc32d903a44808e7b67feeef75cd2 (table test-table [id=a9369413cb414ae489bd75749e7903ee]) successfully deleted
W20260808 18:21:44.080744 23686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:38785 (error 108)
I20260808 18:21:44.081017 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 23993
I20260808 18:21:44.083621 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.083190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:38785 (local address 127.0.0.1:51018)
0808 18:21:44.083390 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.083516 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:38785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260808 18:21:44.083971 23686 catalog_manager.cc:4584] TS 6d412dd0e2e24b32accfb26f0291ad6a (127.27.18.193:38785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26adc32d903a44808e7b67feeef75cd2: Network error: Client connection negotiation failed: client connection to 127.27.18.193:38785: connect: Connection refused (error 111)
I20260808 18:21:44.091246 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 24190
I20260808 18:21:44.099577 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 23668
2026-08-08T18:21:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2462 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T18:21:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:44Z Disabled control of system clock
I20260808 18:21:44.133777 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36395
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:36395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:21:44.235687 24445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:44.235895 24445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:44.235942 24445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:44.239657 24445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:44.239702 24445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:44.239717 24445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:44.239749 24445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:44.239763 24445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:44.244047 24445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:36395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:36395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:44.244868 24445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:44.245994 24445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:44.253767 24451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.253778 24452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.253836 24454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:44.254107 24445 server_base.cc:1034] running on GCE node
I20260808 18:21:44.254587 24445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:44.255604 24445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:44.256779 24445 hybrid_clock.cc:648] HybridClock initialized: now 1786213304256752 us; error 40 us; skew 500 ppm
I20260808 18:21:44.258436 24445 webserver.cc:460] Webserver started at http://127.27.18.254:41065/ using document root <none> and password file <none>
I20260808 18:21:44.258941 24445 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:44.259029 24445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:44.259246 24445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:44.261101 24445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "b86fb655a1614a18b1af191d911dc4e7"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "adff8c9a76163993e0e0ba1e9c20bc1f"
server_key_iv: "b5750782bbf2e254fa6ca8b49f95f6fd"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.261643 24445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "b86fb655a1614a18b1af191d911dc4e7"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "adff8c9a76163993e0e0ba1e9c20bc1f"
server_key_iv: "b5750782bbf2e254fa6ca8b49f95f6fd"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.265000 24445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:21:44.267108 24463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.268199 24445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:44.268342 24445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "b86fb655a1614a18b1af191d911dc4e7"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "adff8c9a76163993e0e0ba1e9c20bc1f"
server_key_iv: "b5750782bbf2e254fa6ca8b49f95f6fd"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.268441 24445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:44.280856 24445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:44.281422 24445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:44.281631 24445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:44.288453 24544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:36395 every 8 connection(s)
I20260808 18:21:44.288458 24445 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:36395
I20260808 18:21:44.289309 24445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:44.292099 24545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:44.297176 24545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7: Bootstrap starting.
I20260808 18:21:44.298732 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 24445
I20260808 18:21:44.298890 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:44.299192 27723 external_mini_cluster.cc:1428] Setting key 87d5a6b05c3c13b9caca9034b60a9635
I20260808 18:21:44.299542 24545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:44.300355 24545 log.cc:824] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:44.302465 24545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7: No bootstrap required, opened a new log
I20260808 18:21:44.306528 24545 raft_consensus.cc:348] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86fb655a1614a18b1af191d911dc4e7" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36395 } }
I20260808 18:21:44.306784 24545 raft_consensus.cc:374] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:44.306854 24545 raft_consensus.cc:729] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86fb655a1614a18b1af191d911dc4e7, State: Initialized, Role: FOLLOWER
I20260808 18:21:44.307276 24545 consensus_queue.cc:260] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86fb655a1614a18b1af191d911dc4e7" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36395 } }
I20260808 18:21:44.307487 24545 raft_consensus.cc:388] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:44.307567 24545 raft_consensus.cc:482] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:44.307664 24545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:44.308521 24545 raft_consensus.cc:504] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86fb655a1614a18b1af191d911dc4e7" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36395 } }
I20260808 18:21:44.308565 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.300253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.27.18.254:36395)
0808 18:21:44.300889 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.300904 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.300936 (+ 32us) server_negotiation.cc:407] Connection header received
0808 18:21:44.301726 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.301751 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.302127 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.302466 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:44.302856 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.304268 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.305111 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.305835 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.305998 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.306720 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:44.306726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:44.306740 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:44.307193 (+ 453us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:44.307255 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:44.307279 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:44.307434 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.307610 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.307874 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":117,"threads_started":1}
I20260808 18:21:44.308926 24545 leader_election.cc:302] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b86fb655a1614a18b1af191d911dc4e7; no voters:
I20260808 18:21:44.309285 24545 leader_election.cc:288] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:44.309382 24553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:44.309593 24553 raft_consensus.cc:686] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [term 1 LEADER]: Becoming Leader. State: Replica: b86fb655a1614a18b1af191d911dc4e7, State: Running, Role: LEADER
I20260808 18:21:44.309861 24553 consensus_queue.cc:237] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86fb655a1614a18b1af191d911dc4e7" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36395 } }
I20260808 18:21:44.310343 24545 sys_catalog.cc:564] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:44.311363 24554 sys_catalog.cc:455] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b86fb655a1614a18b1af191d911dc4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86fb655a1614a18b1af191d911dc4e7" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36395 } } }
I20260808 18:21:44.311419 24555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b86fb655a1614a18b1af191d911dc4e7. Latest consensus state: current_term: 1 leader_uuid: "b86fb655a1614a18b1af191d911dc4e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b86fb655a1614a18b1af191d911dc4e7" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 36395 } } }
I20260808 18:21:44.311496 24554 sys_catalog.cc:458] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:44.311512 24555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:44.311817 24566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:44.314843 24566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:44.319962 24566 catalog_manager.cc:1347] Generated new cluster ID: 16e0a78597594f54b8373f39989ffe26
I20260808 18:21:44.320015 24566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:44.352208 24566 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:44.353355 24566 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:44.359525 24566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7: Generated new TSK 0
I20260808 18:21:44.360273 24566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:44.373140 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36395
--builtin_ntp_servers=127.27.18.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:21:44.478345 24577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:44.478590 24577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:44.478672 24577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:44.482240 24577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:44.482303 24577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:44.482384 24577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:44.486726 24577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:44.487697 24577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:44.488889 24577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:44.497731 24582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:44.497776 24577 server_base.cc:1034] running on GCE node
W20260808 18:21:44.497782 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.497925 24583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:44.498318 24577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:44.498939 24577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:44.500089 24577 hybrid_clock.cc:648] HybridClock initialized: now 1786213304500069 us; error 30 us; skew 500 ppm
I20260808 18:21:44.501629 24577 webserver.cc:460] Webserver started at http://127.27.18.193:33955/ using document root <none> and password file <none>
I20260808 18:21:44.502149 24577 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:44.502210 24577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:44.502463 24577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:44.504451 24577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "87244c9a5b7e4efda09158d1b4fef9c5"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "ca896d4e84b178a9f1df1604746d18ba"
server_key_iv: "f381e4e2203b466404fc3018cd94834b"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.504977 24577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "87244c9a5b7e4efda09158d1b4fef9c5"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "ca896d4e84b178a9f1df1604746d18ba"
server_key_iv: "f381e4e2203b466404fc3018cd94834b"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.508329 24577 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 18:21:44.510561 24595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.511708 24577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:44.511828 24577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "87244c9a5b7e4efda09158d1b4fef9c5"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "ca896d4e84b178a9f1df1604746d18ba"
server_key_iv: "f381e4e2203b466404fc3018cd94834b"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.511927 24577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:44.524078 24577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:44.524658 24577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:44.524856 24577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:44.525373 24577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:44.526199 24577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:44.526269 24577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.526346 24577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:44.526382 24577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.534703 24577 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:33651
I20260808 18:21:44.534729 24760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:33651 every 8 connection(s)
I20260808 18:21:44.535526 24577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:44.540092 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 24577
I20260808 18:21:44.540241 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:21:44.540519 27723 external_mini_cluster.cc:1428] Setting key e0a34764ae9b5283dbf53c2e5e473290
I20260808 18:21:44.542331 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36395
--builtin_ntp_servers=127.27.18.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:21:44.544792 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.537110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:58919 (local address 127.27.18.254:36395)
0808 18:21:44.537252 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.537256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.538354 (+ 1098us) server_negotiation.cc:407] Connection header received
0808 18:21:44.539711 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.539714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.539770 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.539864 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:44.540752 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.541404 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.542730 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.542958 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.543068 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.544264 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:44.544270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:44.544274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:44.544291 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:44.544317 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:44.544329 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:44.544391 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.544634 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.544686 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:21:44.545863 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.537526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36395 (local address 127.27.18.193:58919)
0808 18:21:44.538175 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.538211 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:44.539517 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:44.540021 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:44.540027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:44.540098 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:44.540609 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.540627 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.541536 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.541539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.542575 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.542596 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.543106 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.543110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.543271 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.543937 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:44.543939 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:44.543976 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:44.544064 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:44.544466 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:44.544478 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:44.544485 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 18:21:44.544611 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 18:21:44.544904 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":120,"threads_started":1}
I20260808 18:21:44.546933 24761 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36395
I20260808 18:21:44.547183 24761 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:44.547806 24761 heartbeater.cc:511] Master 127.27.18.254:36395 requested a full tablet report, sending...
I20260808 18:21:44.549342 24487 ts_manager.cc:194] Registered new tserver with Master: 87244c9a5b7e4efda09158d1b4fef9c5 (127.27.18.193:33651)
I20260808 18:21:44.550413 24487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.193:58919
W20260808 18:21:44.649102 24764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:44.649300 24764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:44.649333 24764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:44.653169 24764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:44.653209 24764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:44.653256 24764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:44.657486 24764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:44.658294 24764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:44.659458 24764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:44.668382 24776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.668387 24774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.668362 24773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:44.668700 24764 server_base.cc:1034] running on GCE node
I20260808 18:21:44.669061 24764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:44.669660 24764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:44.670825 24764 hybrid_clock.cc:648] HybridClock initialized: now 1786213304670811 us; error 23 us; skew 500 ppm
I20260808 18:21:44.672706 24764 webserver.cc:460] Webserver started at http://127.27.18.194:37717/ using document root <none> and password file <none>
I20260808 18:21:44.673235 24764 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:44.673295 24764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:44.673523 24764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:44.675441 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "de4c7e1e88274d969dc76000724a94c4"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "64f4a9292c58e140b5f24dfa345b4e51"
server_key_iv: "ec1e52253ba202f4c0d984f89f79eddf"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.676002 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "de4c7e1e88274d969dc76000724a94c4"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "64f4a9292c58e140b5f24dfa345b4e51"
server_key_iv: "ec1e52253ba202f4c0d984f89f79eddf"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.679335 24764 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:21:44.681622 24785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.682698 24764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:44.682830 24764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "de4c7e1e88274d969dc76000724a94c4"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "64f4a9292c58e140b5f24dfa345b4e51"
server_key_iv: "ec1e52253ba202f4c0d984f89f79eddf"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.682936 24764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:44.695601 24764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:44.696151 24764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:44.696328 24764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:44.696843 24764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:44.697669 24764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:44.697734 24764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.697803 24764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:44.697849 24764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.706434 24764 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:33077
I20260808 18:21:44.706457 24953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:33077 every 8 connection(s)
I20260808 18:21:44.707224 24764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:44.710031 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 24764
I20260808 18:21:44.710181 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:21:44.710549 27723 external_mini_cluster.cc:1428] Setting key 4ede83030672cb6a9fd867d01e71647b
I20260808 18:21:44.712870 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36395
--builtin_ntp_servers=127.27.18.212:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:21:44.716818 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.708895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:49075 (local address 127.27.18.254:36395)
0808 18:21:44.709046 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.709051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.710074 (+ 1023us) server_negotiation.cc:407] Connection header received
0808 18:21:44.711777 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.711780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.711834 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.711930 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:44.712764 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.713522 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.714717 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.714951 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.715070 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.716222 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:44.716228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:44.716234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:44.716251 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:44.716278 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:44.716293 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:44.716367 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.716532 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.716603 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:21:44.717428 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.709262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36395 (local address 127.27.18.194:49075)
0808 18:21:44.709911 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.709966 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:44.711590 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:44.712073 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:44.712079 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:44.712151 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:44.712620 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.712631 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.713648 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.713650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.714521 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.714530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.715106 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.715110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.715271 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.715963 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:44.715964 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:44.715999 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:44.716055 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:44.716387 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:44.716395 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:44.716400 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 18:21:44.716501 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:21:44.716690 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":117,"threads_started":1}
I20260808 18:21:44.718570 24954 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36395
I20260808 18:21:44.718825 24954 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:44.719435 24954 heartbeater.cc:511] Master 127.27.18.254:36395 requested a full tablet report, sending...
I20260808 18:21:44.720505 24487 ts_manager.cc:194] Registered new tserver with Master: de4c7e1e88274d969dc76000724a94c4 (127.27.18.194:33077)
I20260808 18:21:44.721045 24487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.194:49075
W20260808 18:21:44.821422 24958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:44.821667 24958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:44.821731 24958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:44.825356 24958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:44.825426 24958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:44.825506 24958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:44.829864 24958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:36395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:44.830780 24958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:44.831964 24958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:21:44.841287 24958 server_base.cc:1034] running on GCE node
W20260808 18:21:44.841298 24965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.841292 24967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:44.841315 24964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:44.841948 24958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:44.842519 24958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:44.843703 24958 hybrid_clock.cc:648] HybridClock initialized: now 1786213304843677 us; error 39 us; skew 500 ppm
I20260808 18:21:44.845587 24958 webserver.cc:460] Webserver started at http://127.27.18.195:39189/ using document root <none> and password file <none>
I20260808 18:21:44.846189 24958 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:44.846282 24958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:44.846510 24958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:44.848428 24958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "7a27f20ff8d749febde80f4c6def752c"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "79c6b593147941d37df72494ebea9923"
server_key_iv: "42fbc4790e7b77c86de2d39a83c7d593"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.848949 24958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "7a27f20ff8d749febde80f4c6def752c"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "79c6b593147941d37df72494ebea9923"
server_key_iv: "42fbc4790e7b77c86de2d39a83c7d593"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.852449 24958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:21:44.854610 24976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.855813 24958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:44.855942 24958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "7a27f20ff8d749febde80f4c6def752c"
format_stamp: "Formatted at 2026-08-08 18:21:44 on dist-test-slave-x4qh"
server_key: "79c6b593147941d37df72494ebea9923"
server_key_iv: "42fbc4790e7b77c86de2d39a83c7d593"
server_key_version: "encryptionkey@0"
I20260808 18:21:44.856047 24958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:44.874236 24958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:44.874854 24958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:44.875039 24958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:44.875667 24958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:44.876628 24958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:44.876700 24958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.876771 24958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:44.876816 24958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:44.885485 24958 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42467
I20260808 18:21:44.885524 25134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42467 every 8 connection(s)
I20260808 18:21:44.886307 24958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:44.890558 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 24958
I20260808 18:21:44.890673 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:21:44.890986 27723 external_mini_cluster.cc:1428] Setting key 53ec9fb93e536bf957dd0ebec1c0b309
I20260808 18:21:44.895087 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.887987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:59719 (local address 127.27.18.254:36395)
0808 18:21:44.888135 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.888140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.889069 (+ 929us) server_negotiation.cc:407] Connection header received
0808 18:21:44.890340 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.890342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.890387 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.890456 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:44.891552 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.892244 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.893331 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.893570 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.893669 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.894603 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:44.894606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:44.894609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:44.894621 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:44.894642 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:44.894658 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:44.894731 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.894892 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.894950 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 18:21:44.895903 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.888372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:36395 (local address 127.27.18.195:59719)
0808 18:21:44.888939 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.888974 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:44.890132 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:44.890661 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:44.890670 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:44.890763 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:44.891359 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.891423 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.892371 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.892375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.893204 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.893213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.893715 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.893719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.893892 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.894371 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:44.894373 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:44.894408 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:44.894462 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:44.894745 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:44.894756 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:44.894766 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 18:21:44.894890 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 18:21:44.895136 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260808 18:21:44.896916 25136 heartbeater.cc:347] Connected to a master server at 127.27.18.254:36395
I20260808 18:21:44.897122 25136 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:44.897580 25136 heartbeater.cc:511] Master 127.27.18.254:36395 requested a full tablet report, sending...
I20260808 18:21:44.898494 24487 ts_manager.cc:194] Registered new tserver with Master: 7a27f20ff8d749febde80f4c6def752c (127.27.18.195:42467)
I20260808 18:21:44.898970 24487 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.195:59719
I20260808 18:21:44.904606 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:44.909559 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.906366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.27.18.254:36395)
0808 18:21:44.906491 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.906495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.906662 (+ 167us) server_negotiation.cc:407] Connection header received
0808 18:21:44.906745 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.906747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.906778 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.906842 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:44.907021 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.907501 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.908231 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.908413 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.908480 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.909171 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:44.909174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:44.909175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:44.909186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:44.909202 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:44.909209 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:44.909307 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.909363 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.909425 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 18:21:44.912160 24487 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:44.915077 24487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:44.928666 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.922541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33077 (local address 127.0.0.1:44112)
0808 18:21:44.923330 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.923393 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:44.923508 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:44.924261 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:44.924266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:44.924285 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:44.924379 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.924384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.925715 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.925719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.926927 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.926936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.928102 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.928106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.928228 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.928268 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 18:21:44.928399 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 18:21:44.928526 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":22,"thread_start_us":58,"threads_started":1}
I20260808 18:21:44.928725 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.922665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.27.18.194:33077)
0808 18:21:44.923790 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.923802 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.923846 (+ 44us) server_negotiation.cc:407] Connection header received
0808 18:21:44.923927 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.923932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.924112 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.924261 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:44.924493 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.925598 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.927063 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.927967 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.928078 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.928242 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.928442 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.928550 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018,"thread_start_us":92,"threads_started":1}
I20260808 18:21:44.929343 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.923115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33651 (local address 127.0.0.1:33268)
0808 18:21:44.923777 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.923790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:44.923883 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:44.925194 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:44.925197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:44.925213 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:44.925288 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.925293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.926469 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.926471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.927670 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.927682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.928942 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.928946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.929019 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.929036 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:21:44.929123 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:21:44.929188 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":36,"thread_start_us":86,"threads_started":1}
I20260808 18:21:44.929665 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.923263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.27.18.193:33651)
0808 18:21:44.924584 (+ 1321us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.924595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.924616 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:21:44.924740 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.924745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.924918 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.925058 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:44.925420 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.926345 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.927826 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.928601 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.928699 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.928880 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.929156 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.929239 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215,"thread_start_us":186,"threads_started":1}
I20260808 18:21:44.929975 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.923153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.27.18.195:42467)
0808 18:21:44.923555 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:44.923569 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:44.923896 (+ 327us) server_negotiation.cc:407] Connection header received
0808 18:21:44.923959 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:44.923963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:44.924112 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:44.924261 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:44.924636 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.926518 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.927717 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.928361 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.928458 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.928623 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:44.929588 (+ 965us) server_negotiation.cc:299] Negotiation successful
0808 18:21:44.929680 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":123,"threads_started":1}
I20260808 18:21:44.930231 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:44.922648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42467 (local address 127.0.0.1:39270)
0808 18:21:44.923684 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:44.923697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:44.923778 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:44.924473 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:44.924475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:44.924486 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:44.924531 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.924534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.926637 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.926639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.927599 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:44.927609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:44.928688 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:44.928692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:44.929074 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:44.929099 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 18:21:44.929172 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 18:21:44.929236 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260808 18:21:44.931803 24862 tablet_service.cc:1467] Processing CreateTablet for tablet 6ea726a306234638a8624331d96c91cd (DEFAULT_TABLE table=test-table [id=e493a479c9844c3b9e1b4275550f4f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:44.932399 24663 tablet_service.cc:1467] Processing CreateTablet for tablet 6ea726a306234638a8624331d96c91cd (DEFAULT_TABLE table=test-table [id=e493a479c9844c3b9e1b4275550f4f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:44.932854 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ea726a306234638a8624331d96c91cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:44.932595 25044 tablet_service.cc:1467] Processing CreateTablet for tablet 6ea726a306234638a8624331d96c91cd (DEFAULT_TABLE table=test-table [id=e493a479c9844c3b9e1b4275550f4f94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:44.933274 24663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ea726a306234638a8624331d96c91cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:44.933432 25044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ea726a306234638a8624331d96c91cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:44.939167 25165 tablet_bootstrap.cc:492] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Bootstrap starting.
I20260808 18:21:44.940032 25167 tablet_bootstrap.cc:492] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: Bootstrap starting.
I20260808 18:21:44.940033 25166 tablet_bootstrap.cc:492] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Bootstrap starting.
I20260808 18:21:44.941195 25165 tablet_bootstrap.cc:654] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:44.941854 25165 log.cc:824] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:44.941936 25166 tablet_bootstrap.cc:654] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:44.941936 25167 tablet_bootstrap.cc:654] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:44.942763 25166 log.cc:824] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:44.942763 25167 log.cc:824] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:44.943823 25165 tablet_bootstrap.cc:492] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: No bootstrap required, opened a new log
I20260808 18:21:44.944000 25165 ts_tablet_manager.cc:1397] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:21:44.944602 25166 tablet_bootstrap.cc:492] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: No bootstrap required, opened a new log
I20260808 18:21:44.944788 25166 ts_tablet_manager.cc:1397] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:21:44.945111 25167 tablet_bootstrap.cc:492] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: No bootstrap required, opened a new log
I20260808 18:21:44.945274 25167 ts_tablet_manager.cc:1397] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:44.947737 25166 raft_consensus.cc:348] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:44.947964 25166 raft_consensus.cc:374] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:44.948015 25166 raft_consensus.cc:729] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c7e1e88274d969dc76000724a94c4, State: Initialized, Role: FOLLOWER
I20260808 18:21:44.948213 25165 raft_consensus.cc:348] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:44.948452 25165 raft_consensus.cc:374] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:44.948518 25165 raft_consensus.cc:729] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a27f20ff8d749febde80f4c6def752c, State: Initialized, Role: FOLLOWER
I20260808 18:21:44.948446 25166 consensus_queue.cc:260] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:44.948905 25165 consensus_queue.cc:260] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:44.949205 25167 raft_consensus.cc:348] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:44.949401 25166 ts_tablet_manager.cc:1428] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 18:21:44.949441 25167 raft_consensus.cc:374] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:44.949487 25167 raft_consensus.cc:729] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87244c9a5b7e4efda09158d1b4fef9c5, State: Initialized, Role: FOLLOWER
I20260808 18:21:44.949715 24954 heartbeater.cc:503] Master 127.27.18.254:36395 was elected leader, sending a full tablet report...
I20260808 18:21:44.949823 25165 ts_tablet_manager.cc:1428] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:21:44.949862 25167 consensus_queue.cc:260] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:44.950130 25136 heartbeater.cc:503] Master 127.27.18.254:36395 was elected leader, sending a full tablet report...
I20260808 18:21:44.950693 25167 ts_tablet_manager.cc:1428] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:21:44.950982 24761 heartbeater.cc:503] Master 127.27.18.254:36395 was elected leader, sending a full tablet report...
W20260808 18:21:44.958118 24955 tablet.cc:2369] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:21:45.036607 24762 tablet.cc:2369] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:45.052835 25174 raft_consensus.cc:482] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:45.052994 25174 raft_consensus.cc:504] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:45.054026 25174 leader_election.cc:288] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a27f20ff8d749febde80f4c6def752c (127.27.18.195:42467), 87244c9a5b7e4efda09158d1b4fef9c5 (127.27.18.193:33651)
I20260808 18:21:45.057560 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.054261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42467 (local address 127.27.18.194:46803)
0808 18:21:45.054411 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:45.054428 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:45.054523 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:45.054790 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:45.054793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:45.054810 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:45.054929 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:45.054935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.055697 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.055701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:45.056439 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:45.056447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.057093 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.057096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:45.057197 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:45.057221 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 18:21:45.057319 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:21:45.057401 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:21:45.057560 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.054390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:46803 (local address 127.27.18.195:42467)
0808 18:21:45.054522 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:45.054527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:45.054554 (+ 27us) server_negotiation.cc:407] Connection header received
0808 18:21:45.054637 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:45.054641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:45.054690 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:45.054776 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:45.055051 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.055596 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.056584 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.056970 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.057100 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:45.057151 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:45.057329 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 18:21:45.057410 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 18:21:45.058018 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.054261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33651 (local address 127.27.18.194:58693)
0808 18:21:45.054765 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:45.054776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:45.054902 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:45.055163 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:45.055166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:45.055181 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:45.055305 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:45.055310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.055978 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.055981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:45.056782 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:45.056790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.057465 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.057469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:45.057584 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:45.057601 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 18:21:45.057791 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 18:21:45.057870 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":143,"thread_start_us":146,"threads_started":1}
I20260808 18:21:45.058033 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.054382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:58693 (local address 127.27.18.193:33651)
0808 18:21:45.054577 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:45.054583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:45.054865 (+ 282us) server_negotiation.cc:407] Connection header received
0808 18:21:45.055011 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:45.055014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:45.055073 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:45.055151 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:45.055445 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.055875 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.056920 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.057309 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.057465 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:45.057524 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:45.057839 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 18:21:45.057914 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 18:21:45.058260 25069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ea726a306234638a8624331d96c91cd" candidate_uuid: "de4c7e1e88274d969dc76000724a94c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a27f20ff8d749febde80f4c6def752c" is_pre_election: true
I20260808 18:21:45.058588 25069 raft_consensus.cc:2454] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c7e1e88274d969dc76000724a94c4 in term 0.
I20260808 18:21:45.058605 24703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ea726a306234638a8624331d96c91cd" candidate_uuid: "de4c7e1e88274d969dc76000724a94c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" is_pre_election: true
I20260808 18:21:45.058892 24703 raft_consensus.cc:2454] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4c7e1e88274d969dc76000724a94c4 in term 0.
I20260808 18:21:45.059130 24787 leader_election.cc:302] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a27f20ff8d749febde80f4c6def752c, de4c7e1e88274d969dc76000724a94c4; no voters:
I20260808 18:21:45.059449 25174 raft_consensus.cc:2781] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:45.059535 25174 raft_consensus.cc:482] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:45.059582 25174 raft_consensus.cc:3037] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:45.060549 25174 raft_consensus.cc:504] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:45.060926 25174 leader_election.cc:288] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7a27f20ff8d749febde80f4c6def752c (127.27.18.195:42467), 87244c9a5b7e4efda09158d1b4fef9c5 (127.27.18.193:33651)
I20260808 18:21:45.061187 25069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ea726a306234638a8624331d96c91cd" candidate_uuid: "de4c7e1e88274d969dc76000724a94c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a27f20ff8d749febde80f4c6def752c"
I20260808 18:21:45.061252 24703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ea726a306234638a8624331d96c91cd" candidate_uuid: "de4c7e1e88274d969dc76000724a94c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87244c9a5b7e4efda09158d1b4fef9c5"
I20260808 18:21:45.061331 25069 raft_consensus.cc:3037] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:45.061353 24703 raft_consensus.cc:3037] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:45.062565 24703 raft_consensus.cc:2454] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c7e1e88274d969dc76000724a94c4 in term 1.
I20260808 18:21:45.062565 25069 raft_consensus.cc:2454] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4c7e1e88274d969dc76000724a94c4 in term 1.
I20260808 18:21:45.062880 24787 leader_election.cc:302] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a27f20ff8d749febde80f4c6def752c, de4c7e1e88274d969dc76000724a94c4; no voters:
I20260808 18:21:45.063078 25174 raft_consensus.cc:2781] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:45.063273 25174 raft_consensus.cc:686] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 1 LEADER]: Becoming Leader. State: Replica: de4c7e1e88274d969dc76000724a94c4, State: Running, Role: LEADER
I20260808 18:21:45.063608 25174 consensus_queue.cc:237] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } }
I20260808 18:21:45.066581 24484 catalog_manager.cc:5495] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to de4c7e1e88274d969dc76000724a94c4 (127.27.18.194). New cstate: current_term: 1 leader_uuid: "de4c7e1e88274d969dc76000724a94c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de4c7e1e88274d969dc76000724a94c4" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 33077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 } health_report { overall_health: UNKNOWN } } }
I20260808 18:21:45.082029 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.079087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.27.18.194:33077)
0808 18:21:45.079189 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:45.079192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:45.079204 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:21:45.079311 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:45.079313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:45.079359 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:45.079491 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:45.079697 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.080135 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.080879 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:45.081069 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:45.081138 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:45.081387 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:45.081733 (+ 346us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:45.081833 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:45.081906 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 18:21:45.081931 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:21:45.087095 24703 raft_consensus.cc:1261] T 6ea726a306234638a8624331d96c91cd P 87244c9a5b7e4efda09158d1b4fef9c5 [term 1 FOLLOWER]: Refusing update from remote peer de4c7e1e88274d969dc76000724a94c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:45.087126 25069 raft_consensus.cc:1261] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 1 FOLLOWER]: Refusing update from remote peer de4c7e1e88274d969dc76000724a94c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:45.087761 25174 consensus_queue.cc:1035] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a27f20ff8d749febde80f4c6def752c" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:45.087934 25180 consensus_queue.cc:1035] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87244c9a5b7e4efda09158d1b4fef9c5" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 33651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:45.097301 25186 mvcc.cc:205] Tried to move back new op lower bound from 7316329697630928896 to 7316329697542991872. Current Snapshot: MvccSnapshot[applied={T|T < 7316329697630928896 or (T in {7316329697630928896})}]
I20260808 18:21:45.097530 25185 mvcc.cc:205] Tried to move back new op lower bound from 7316329697630928896 to 7316329697542991872. Current Snapshot: MvccSnapshot[applied={T|T < 7316329697630928896}]
I20260808 18:21:45.101994 24486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:45.102111 24486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:45.104288 24486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b86fb655a1614a18b1af191d911dc4e7: Sending DeleteTablet for 3 replicas of tablet 6ea726a306234638a8624331d96c91cd
I20260808 18:21:45.105057 25044 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ea726a306234638a8624331d96c91cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:45 UTC) from {username='slave'} at 127.0.0.1:39270
I20260808 18:21:45.105059 24663 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ea726a306234638a8624331d96c91cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:45 UTC) from {username='slave'} at 127.0.0.1:33268
I20260808 18:21:45.105556 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 24577
I20260808 18:21:45.105695 24862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ea726a306234638a8624331d96c91cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:45 UTC) from {username='slave'} at 127.0.0.1:44112
I20260808 18:21:45.108141 25195 tablet_replica.cc:330] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: stopping tablet replica
I20260808 18:21:45.108462 25195 raft_consensus.cc:2229] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:45.108732 25195 raft_consensus.cc:2258] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:45.110548 25195 ts_tablet_manager.cc:1905] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:45.113895 25195 ts_tablet_manager.cc:1918] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:45.114084 25195 log.cc:1196] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/6ea726a306234638a8624331d96c91cd
I20260808 18:21:45.114542 25195 ts_tablet_manager.cc:1939] T 6ea726a306234638a8624331d96c91cd P 7a27f20ff8d749febde80f4c6def752c: Deleting consensus metadata
I20260808 18:21:45.115624 24466 catalog_manager.cc:4841] TS 7a27f20ff8d749febde80f4c6def752c (127.27.18.195:42467): tablet 6ea726a306234638a8624331d96c91cd (table test-table [id=e493a479c9844c3b9e1b4275550f4f94]) successfully deleted
I20260808 18:21:45.119558 25196 tablet_replica.cc:330] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: stopping tablet replica
I20260808 18:21:45.119818 25196 raft_consensus.cc:2229] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:21:45.120117 25196 raft_consensus.cc:2258] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:45.121249 25196 ts_tablet_manager.cc:1905] T 6ea726a306234638a8624331d96c91cd P de4c7e1e88274d969dc76000724a94c4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 18:21:45.122071 24464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:33651 (error 108)
I20260808 18:21:45.122256 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 24764
I20260808 18:21:45.127758 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.127492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:33651 (local address 127.0.0.1:33282)
0808 18:21:45.127640 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:45.127691 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:33651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 18:21:45.127974 24464 catalog_manager.cc:4584] TS 87244c9a5b7e4efda09158d1b4fef9c5 (127.27.18.193:33651): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ea726a306234638a8624331d96c91cd: Network error: Client connection negotiation failed: client connection to 127.27.18.193:33651: connect: Connection refused (error 111)
I20260808 18:21:45.135766 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 24958
I20260808 18:21:45.135795 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:45.135562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:33077 (local address 127.0.0.1:44126)
0808 18:21:45.135690 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:45.135742 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.194:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:21:45.142163 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 24445
2026-08-08T18:21:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1043 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:45 dist-test-slave-x4qh krb5kdc[25203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:45 dist-test-slave-x4qh krb5kdc[25203](info): set up 2 sockets
Aug 08 18:21:45 dist-test-slave-x4qh krb5kdc[25203](info): commencing operation
krb5kdc: starting...
W20260808 18:21:46.310519 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:46 dist-test-slave-x4qh krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:21:46.423080 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:34825
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:34825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:46.529832 25220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:46.530090 25220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:46.530167 25220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:46.534119 25220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:46.534197 25220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:46.534241 25220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:46.534298 25220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:46.534335 25220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:46.539525 25220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:34825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:34825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:46.540441 25220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:46.541416 25220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:46.549602 25228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:46.549606 25230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:46.549650 25235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:46.550022 25220 server_base.cc:1034] running on GCE node
I20260808 18:21:46.550544 25220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:46.551589 25220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:46.552767 25220 hybrid_clock.cc:648] HybridClock initialized: now 1786213306552744 us; error 34 us; skew 500 ppm
Aug 08 18:21:46 dist-test-slave-x4qh krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213306, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:46.555711 25220 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:21:46.556545 25220 webserver.cc:460] Webserver started at http://127.27.18.254:35129/ using document root <none> and password file <none>
I20260808 18:21:46.557051 25220 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:46.557168 25220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:46.557392 25220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:46.559228 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "c01a03e43ef645e6a9d6d67070dc4655"
format_stamp: "Formatted at 2026-08-08 18:21:46 on dist-test-slave-x4qh"
server_key: "7a3b611354be0c5e27ba8f2c514aa2c7"
server_key_iv: "d5d3988ec5d063244b351cded8d4cdcd"
server_key_version: "encryptionkey@0"
I20260808 18:21:46.559890 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "c01a03e43ef645e6a9d6d67070dc4655"
format_stamp: "Formatted at 2026-08-08 18:21:46 on dist-test-slave-x4qh"
server_key: "7a3b611354be0c5e27ba8f2c514aa2c7"
server_key_iv: "d5d3988ec5d063244b351cded8d4cdcd"
server_key_version: "encryptionkey@0"
I20260808 18:21:46.563264 25220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:21:46.565501 25244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:46.566578 25220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:46.566713 25220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "c01a03e43ef645e6a9d6d67070dc4655"
format_stamp: "Formatted at 2026-08-08 18:21:46 on dist-test-slave-x4qh"
server_key: "7a3b611354be0c5e27ba8f2c514aa2c7"
server_key_iv: "d5d3988ec5d063244b351cded8d4cdcd"
server_key_version: "encryptionkey@0"
I20260808 18:21:46.566826 25220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:46.594714 25220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:46.597697 25220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:46.597887 25220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:46.604780 25220 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:34825
I20260808 18:21:46.604816 25322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:34825 every 8 connection(s)
I20260808 18:21:46.605698 25220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:46.608130 25323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:46.608824 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 25220
I20260808 18:21:46.608961 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:46.609273 27723 external_mini_cluster.cc:1428] Setting key 50114b397e9426740d90a5067b6088ed
I20260808 18:21:46.613535 25323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655: Bootstrap starting.
I20260808 18:21:46.615574 25323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:46.616220 25323 log.cc:824] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655: Log is configured to *not* fsync() on all Append() calls
Aug 08 18:21:46 dist-test-slave-x4qh krb5kdc[25203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:46.618176 25323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655: No bootstrap required, opened a new log
I20260808 18:21:46.621888 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:46.610556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.27.18.254:34825)
0808 18:21:46.610946 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:46.610958 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:46.610988 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:21:46.611601 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:46.611618 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:46.611893 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:46.612195 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:46.612928 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:46.613873 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:46.614727 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:46.615194 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:46.615355 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:46.617817 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:46.617822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:46.617833 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:46.617856 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:46.619414 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:46.619879 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:46.619882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:46.619886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:46.619977 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:46.620290 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:46.620292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:46.620293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:46.620611 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:46.620737 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:46.621004 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 18:21:46.621266 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
I20260808 18:21:46.621903 25323 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a03e43ef645e6a9d6d67070dc4655" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34825 } }
I20260808 18:21:46.622184 25323 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:46.622264 25323 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01a03e43ef645e6a9d6d67070dc4655, State: Initialized, Role: FOLLOWER
I20260808 18:21:46.622670 25323 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a03e43ef645e6a9d6d67070dc4655" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34825 } }
I20260808 18:21:46.622778 25323 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:46.622838 25323 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:46.622928 25323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:46.623792 25323 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a03e43ef645e6a9d6d67070dc4655" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34825 } }
I20260808 18:21:46.624136 25323 leader_election.cc:302] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01a03e43ef645e6a9d6d67070dc4655; no voters:
I20260808 18:21:46.624439 25323 leader_election.cc:288] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:46.624539 25330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:46.624779 25330 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [term 1 LEADER]: Becoming Leader. State: Replica: c01a03e43ef645e6a9d6d67070dc4655, State: Running, Role: LEADER
I20260808 18:21:46.625079 25330 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a03e43ef645e6a9d6d67070dc4655" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34825 } }
I20260808 18:21:46.625550 25323 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:46.626659 25334 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01a03e43ef645e6a9d6d67070dc4655. Latest consensus state: current_term: 1 leader_uuid: "c01a03e43ef645e6a9d6d67070dc4655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a03e43ef645e6a9d6d67070dc4655" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34825 } } }
I20260808 18:21:46.626660 25333 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01a03e43ef645e6a9d6d67070dc4655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a03e43ef645e6a9d6d67070dc4655" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 34825 } } }
I20260808 18:21:46.626866 25333 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:46.626827 25334 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:46.627221 25347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:46.629823 25347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:46.634917 25347 catalog_manager.cc:1347] Generated new cluster ID: f2c4977f56b9449896179075206f908d
I20260808 18:21:46.634985 25347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:46.646183 25347 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:46.647264 25347 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:46.654080 25347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655: Generated new TSK 0
I20260808 18:21:46.654739 25347 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
Entry for principal kudu/127.27.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab.
I20260808 18:21:46.689930 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34825
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:46.792507 25365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:46.792721 25365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:46.792758 25365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:46.796465 25365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:46.796506 25365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:46.796554 25365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:46.802496 25365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.193.keytab
--principal=kudu/127.27.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:46.803264 25365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:46.804170 25365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:46.812903 25372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:46.812908 25373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:46.813047 25365 server_base.cc:1034] running on GCE node
W20260808 18:21:46.812889 25375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:46.813491 25365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:46.814069 25365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:46.815258 25365 hybrid_clock.cc:648] HybridClock initialized: now 1786213306815244 us; error 22 us; skew 500 ppm
Aug 08 18:21:46 dist-test-slave-x4qh krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213306, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:46.817842 25365 init.cc:377] Logged in from keytab as kudu/127.27.18.193@KRBTEST.COM (short username kudu)
I20260808 18:21:46.818634 25365 webserver.cc:460] Webserver started at http://127.27.18.193:37937/ using document root <none> and password file <none>
I20260808 18:21:46.819093 25365 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:46.819149 25365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:46.819296 25365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:46.821144 25365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "3bef5cc8849c4cd8bb01da41d652dc6d"
format_stamp: "Formatted at 2026-08-08 18:21:46 on dist-test-slave-x4qh"
server_key: "25a065584dc26dd336c032a0b513bdd5"
server_key_iv: "a71d525d2b4ab3d37a584454a13fed3b"
server_key_version: "encryptionkey@0"
I20260808 18:21:46.821624 25365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "3bef5cc8849c4cd8bb01da41d652dc6d"
format_stamp: "Formatted at 2026-08-08 18:21:46 on dist-test-slave-x4qh"
server_key: "25a065584dc26dd336c032a0b513bdd5"
server_key_iv: "a71d525d2b4ab3d37a584454a13fed3b"
server_key_version: "encryptionkey@0"
I20260808 18:21:46.825008 25365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:21:46.827265 25385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:46.828382 25365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:46.828482 25365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "3bef5cc8849c4cd8bb01da41d652dc6d"
format_stamp: "Formatted at 2026-08-08 18:21:46 on dist-test-slave-x4qh"
server_key: "25a065584dc26dd336c032a0b513bdd5"
server_key_iv: "a71d525d2b4ab3d37a584454a13fed3b"
server_key_version: "encryptionkey@0"
I20260808 18:21:46.828559 25365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:46.850261 25365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:46.852679 25365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:46.852823 25365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:46.853257 25365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:46.853935 25365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:46.853984 25365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:46.854041 25365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:46.854061 25365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:46.862592 25365 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:34131
I20260808 18:21:46.862619 25559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:34131 every 8 connection(s)
I20260808 18:21:46.863487 25365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:46.865543 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 25365
I20260808 18:21:46.865680 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:21:46.866032 27723 external_mini_cluster.cc:1428] Setting key 0f8a4f7267e847f91cea188a9f3997ff
Aug 08 18:21:46 dist-test-slave-x4qh krb5kdc[25203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213306, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.193@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.194@KRBTEST.COM; defaulting to no policy
I20260808 18:21:46.877898 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:46.865251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:49053 (local address 127.27.18.254:34825)
0808 18:21:46.865440 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:46.865446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:46.866328 (+ 882us) server_negotiation.cc:407] Connection header received
0808 18:21:46.867918 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:46.867920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:46.867959 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:46.868013 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:46.869395 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:46.869950 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:46.871587 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:46.871794 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:46.871872 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:46.874494 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:46.874498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:46.874500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:46.874521 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:46.875940 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:46.876423 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:46.876429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:46.876430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:46.876509 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:46.876858 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:46.876863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:46.876865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:46.877032 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:46.877147 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:46.877614 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 18:21:46.877734 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 18:21:46.878561 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:46.865630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34825 (local address 127.27.18.193:49053)
0808 18:21:46.866164 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:46.866218 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:46.867702 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:46.868164 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:46.868173 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:46.868640 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:46.869237 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:46.869257 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:46.870078 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:46.870083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:46.871477 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:46.871491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:46.871942 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:46.871949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:46.872159 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:46.872702 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:46.872725 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:46.874333 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:46.876057 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:46.876062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:46.876075 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:46.876322 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:46.876603 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:46.876606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:46.876608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:46.876767 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:46.877147 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:46.877152 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:46.877397 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 18:21:46.877570 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 18:21:46.877865 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.194@KRBTEST.COM" created.
I20260808 18:21:46.879696 25560 heartbeater.cc:347] Connected to a master server at 127.27.18.254:34825
I20260808 18:21:46.879928 25560 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:46.880465 25560 heartbeater.cc:511] Master 127.27.18.254:34825 requested a full tablet report, sending...
I20260808 18:21:46.882022 25269 ts_manager.cc:194] Registered new tserver with Master: 3bef5cc8849c4cd8bb01da41d652dc6d (127.27.18.193:34131)
I20260808 18:21:46.883688 25269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.193@KRBTEST.COM'} at 127.27.18.193:49053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
Entry for principal kudu/127.27.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab.
I20260808 18:21:46.894552 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34825
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:46.998646 25567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:46.998893 25567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:46.998965 25567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:47.002667 25567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:47.002744 25567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:47.002823 25567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:47.007637 25567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.194.keytab
--principal=kudu/127.27.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:47.008522 25567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:47.009603 25567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:47.018362 25577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:47.018395 25573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:47.018411 25575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:47.018621 25567 server_base.cc:1034] running on GCE node
I20260808 18:21:47.019096 25567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:47.019735 25567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:47.020890 25567 hybrid_clock.cc:648] HybridClock initialized: now 1786213307020864 us; error 44 us; skew 500 ppm
Aug 08 18:21:47 dist-test-slave-x4qh krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213307, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:47.023495 25567 init.cc:377] Logged in from keytab as kudu/127.27.18.194@KRBTEST.COM (short username kudu)
I20260808 18:21:47.024263 25567 webserver.cc:460] Webserver started at http://127.27.18.194:46433/ using document root <none> and password file <none>
I20260808 18:21:47.024780 25567 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:47.024878 25567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:47.025092 25567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:47.027089 25567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "d08d55bcc13c4697a8869a3653e9262f"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "b2f558e5eb32ded2d5ff547af38f8bb3"
server_key_iv: "1ae40aa91b7615e0d4227dabcb83358c"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.027676 25567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "d08d55bcc13c4697a8869a3653e9262f"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "b2f558e5eb32ded2d5ff547af38f8bb3"
server_key_iv: "1ae40aa91b7615e0d4227dabcb83358c"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.030901 25567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:21:47.033084 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.034204 25567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:21:47.034333 25567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "d08d55bcc13c4697a8869a3653e9262f"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "b2f558e5eb32ded2d5ff547af38f8bb3"
server_key_iv: "1ae40aa91b7615e0d4227dabcb83358c"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.034435 25567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:47.049698 25567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:47.052189 25567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:47.052357 25567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:47.052834 25567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:47.053575 25567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:47.053645 25567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.053707 25567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:47.053750 25567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.061928 25567 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:41931
I20260808 18:21:47.061949 25752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:41931 every 8 connection(s)
I20260808 18:21:47.062641 25567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:47.070338 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 25567
I20260808 18:21:47.070492 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:21:47.070796 27723 external_mini_cluster.cc:1428] Setting key 98df72cfc118f4f8ffd57e50d9a5a199
Aug 08 18:21:47 dist-test-slave-x4qh krb5kdc[25203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213307, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.194@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:47.075332 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.064304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:58721 (local address 127.27.18.254:34825)
0808 18:21:47.064436 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.064440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.065183 (+ 743us) server_negotiation.cc:407] Connection header received
0808 18:21:47.066313 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.066315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.066357 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.066424 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:47.067383 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.067872 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.068937 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.069130 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.069200 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.071743 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:47.071746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:47.071748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:47.071770 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:47.073284 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:47.073751 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:47.073757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:47.073761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:47.073863 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:47.074193 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:47.074199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:47.074203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:47.074446 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:47.074581 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.075050 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.075170 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 18:21:47.076028 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.064613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34825 (local address 127.27.18.194:58721)
0808 18:21:47.064998 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.065067 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.066142 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.066560 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.066565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.066837 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:47.067241 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.067252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.067973 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.067975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.068835 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.068845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.069263 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.069267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.069400 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.069861 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:47.069877 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:47.071572 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:47.073399 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:47.073403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:47.073413 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:47.073658 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:47.073962 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:47.073964 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:47.073965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:47.074105 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:47.074582 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:47.074589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:47.074825 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.075003 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.075284 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
I20260808 18:21:47.077070 25753 heartbeater.cc:347] Connected to a master server at 127.27.18.254:34825
I20260808 18:21:47.077329 25753 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:47.077860 25753 heartbeater.cc:511] Master 127.27.18.254:34825 requested a full tablet report, sending...
I20260808 18:21:47.078926 25269 ts_manager.cc:194] Registered new tserver with Master: d08d55bcc13c4697a8869a3653e9262f (127.27.18.194:41931)
I20260808 18:21:47.079721 25269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.194@KRBTEST.COM'} at 127.27.18.194:58721
WARNING: no policy specified for kudu/127.27.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
Entry for principal kudu/127.27.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab.
I20260808 18:21:47.098393 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34825
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:39309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:47.201944 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:47.202152 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:47.202206 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:47.206030 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:47.206072 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:47.206120 25759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:47.210604 25759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:39309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.195.keytab
--principal=kudu/127.27.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:34825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:47.211494 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:47.212299 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:47.220873 25768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:47.220885 25765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:47.220885 25766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:47.221074 25759 server_base.cc:1034] running on GCE node
I20260808 18:21:47.221537 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:47.222087 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:47.223261 25759 hybrid_clock.cc:648] HybridClock initialized: now 1786213307223242 us; error 36 us; skew 500 ppm
Aug 08 18:21:47 dist-test-slave-x4qh krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213307, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:47.225788 25759 init.cc:377] Logged in from keytab as kudu/127.27.18.195@KRBTEST.COM (short username kudu)
I20260808 18:21:47.226504 25759 webserver.cc:460] Webserver started at http://127.27.18.195:36223/ using document root <none> and password file <none>
I20260808 18:21:47.227031 25759 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:47.227115 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:47.227315 25759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:47.229154 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "c0c473e0eafd1a51fe2480f7e2f559a0"
server_key_iv: "978c26a97fa5cf763b2ffa2edcd930f3"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.229676 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "c0c473e0eafd1a51fe2480f7e2f559a0"
server_key_iv: "978c26a97fa5cf763b2ffa2edcd930f3"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.233212 25759 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:21:47.235306 25779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.236299 25759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:47.236435 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "c0c473e0eafd1a51fe2480f7e2f559a0"
server_key_iv: "978c26a97fa5cf763b2ffa2edcd930f3"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.236532 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:47.245662 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:47.248092 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:47.248266 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:47.248755 25759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:47.249430 25759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:47.249493 25759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.249552 25759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:47.249593 25759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.257761 25759 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:42043
I20260808 18:21:47.257794 25954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:42043 every 8 connection(s)
I20260808 18:21:47.258502 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:47.263681 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 25759
I20260808 18:21:47.263833 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:21:47.264122 27723 external_mini_cluster.cc:1428] Setting key eaee59cac0d7307bd40eaaddc8df738a
Aug 08 18:21:47 dist-test-slave-x4qh krb5kdc[25203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213307, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.195@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:21:47.270903 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.260041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:50305 (local address 127.27.18.254:34825)
0808 18:21:47.260164 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.260167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.260765 (+ 598us) server_negotiation.cc:407] Connection header received
0808 18:21:47.261881 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.261884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.261941 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.262033 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:47.263011 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.263615 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.265060 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.265288 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.265367 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.267806 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:47.267809 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:47.267811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:47.267834 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:47.269217 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:47.269659 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:47.269661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:47.269662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:47.269734 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:47.270011 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:47.270013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:47.270014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:47.270168 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:47.270240 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.270687 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.270777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:21:47.271585 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.260265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:34825 (local address 127.27.18.195:50305)
0808 18:21:47.260634 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.260669 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.261683 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.262182 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.262188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.262473 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:47.262846 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.262857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.263749 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.263752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.264921 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.264932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.265433 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.265437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.265620 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.266107 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:21:47.266125 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:47.267648 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:47.269340 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:47.269344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:47.269354 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:47.269576 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:47.269815 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:21:47.269817 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:21:47.269818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:21:47.269933 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:21:47.270262 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:47.270267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:47.270492 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.270629 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.270836 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":91,"threads_started":1}
I20260808 18:21:47.272562 25955 heartbeater.cc:347] Connected to a master server at 127.27.18.254:34825
I20260808 18:21:47.272802 25955 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:47.273219 25955 heartbeater.cc:511] Master 127.27.18.254:34825 requested a full tablet report, sending...
I20260808 18:21:47.274140 25269 ts_manager.cc:194] Registered new tserver with Master: 468bb92880cc4fd5bb8a6f62bb0ace3a (127.27.18.195:42043)
I20260808 18:21:47.274658 25269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.18.195@KRBTEST.COM'} at 127.27.18.195:50305
I20260808 18:21:47.277779 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:47.286247 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.279514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.27.18.254:34825)
0808 18:21:47.279596 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.279599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.279833 (+ 234us) server_negotiation.cc:407] Connection header received
0808 18:21:47.279933 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.279937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.279994 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.280076 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:47.280680 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.281086 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.281807 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.282019 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.282111 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.283320 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:47.283323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:47.283326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:47.283354 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:47.284735 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:47.285150 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:47.285154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:47.285155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:47.285234 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:47.285503 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:47.285506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:47.285508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:47.285692 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:47.285825 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.286023 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.286115 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 18:21:47.288602 25269 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:47.291105 25269 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:47.303552 25977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.298621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40684 (local address 127.27.18.195:42043)
0808 18:21:47.299299 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.299327 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.299348 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:21:47.299432 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.299437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.299608 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.299744 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:47.300076 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.300752 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.302097 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.302879 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.302997 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.303134 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.303323 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.303426 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":140,"threads_started":1}
I20260808 18:21:47.303545 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.298102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:42043 (local address 127.0.0.1:40684)
0808 18:21:47.298707 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.298723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.298822 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.299724 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.299727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.299745 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:47.299961 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.299968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.300892 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.300896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.301947 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.301957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.303001 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.303004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.303116 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.303157 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.303291 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.303427 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":32,"thread_start_us":80,"threads_started":1}
I20260808 18:21:47.304247 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.298633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.27.18.194:41931)
0808 18:21:47.299062 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.299073 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.299095 (+ 22us) server_negotiation.cc:407] Connection header received
0808 18:21:47.299171 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.299175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.299354 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.299526 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:47.300122 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.301233 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.302641 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.303425 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.303521 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.303643 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.303996 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.304119 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":124,"threads_started":1}
I20260808 18:21:47.304240 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.297975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:34131 (local address 127.0.0.1:50614)
0808 18:21:47.298647 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.298691 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.298826 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.299696 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.299701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.299735 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:47.299984 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.299989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.301413 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.301415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.302382 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.302391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.303710 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.303714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.303848 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.303886 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.303961 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.304084 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":161,"spinlock_wait_cycles":10112,"thread_start_us":67,"threads_started":1}
I20260808 18:21:47.304240 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.298373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:41931 (local address 127.0.0.1:44324)
0808 18:21:47.298985 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.298998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.299078 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.299696 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.299701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.299737 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:47.299981 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.299989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.301382 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.301385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.302524 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.302533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.303710 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.303714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.303852 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.303889 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.303961 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.304082 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":194,"spinlock_wait_cycles":18176,"thread_start_us":89,"threads_started":1}
I20260808 18:21:47.304973 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.298117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.27.18.193:34131)
0808 18:21:47.299095 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.299106 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.299127 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:21:47.299190 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.299193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.299359 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.299526 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:47.300122 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.301231 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.302528 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.303404 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.303515 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.303644 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.304735 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.304822 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":129,"threads_started":1}
I20260808 18:21:47.306365 25848 tablet_service.cc:1467] Processing CreateTablet for tablet 61572d839fd34bccaa20c080de1c10c7 (DEFAULT_TABLE table=test-table [id=69c0a00d37bf44f1a9a555af767b469f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:47.307193 25848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61572d839fd34bccaa20c080de1c10c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:47.307406 25461 tablet_service.cc:1467] Processing CreateTablet for tablet 61572d839fd34bccaa20c080de1c10c7 (DEFAULT_TABLE table=test-table [id=69c0a00d37bf44f1a9a555af767b469f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:47.307941 25654 tablet_service.cc:1467] Processing CreateTablet for tablet 61572d839fd34bccaa20c080de1c10c7 (DEFAULT_TABLE table=test-table [id=69c0a00d37bf44f1a9a555af767b469f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:47.308360 25461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61572d839fd34bccaa20c080de1c10c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:47.308795 25654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61572d839fd34bccaa20c080de1c10c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:47.312937 25979 tablet_bootstrap.cc:492] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Bootstrap starting.
I20260808 18:21:47.314857 25980 tablet_bootstrap.cc:492] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Bootstrap starting.
I20260808 18:21:47.315733 25983 tablet_bootstrap.cc:492] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: Bootstrap starting.
I20260808 18:21:47.315819 25979 tablet_bootstrap.cc:654] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:47.317092 25979 log.cc:824] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:47.317140 25980 tablet_bootstrap.cc:654] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:47.317368 25983 tablet_bootstrap.cc:654] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:47.318243 25980 log.cc:824] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:47.318456 25983 log.cc:824] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:47.319247 25979 tablet_bootstrap.cc:492] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: No bootstrap required, opened a new log
I20260808 18:21:47.319459 25979 ts_tablet_manager.cc:1397] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 18:21:47.320178 25980 tablet_bootstrap.cc:492] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: No bootstrap required, opened a new log
I20260808 18:21:47.320266 25983 tablet_bootstrap.cc:492] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: No bootstrap required, opened a new log
I20260808 18:21:47.320344 25980 ts_tablet_manager.cc:1397] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 18:21:47.320397 25983 ts_tablet_manager.cc:1397] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:47.322732 25983 raft_consensus.cc:348] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.322901 25983 raft_consensus.cc:374] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:47.322937 25983 raft_consensus.cc:729] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bef5cc8849c4cd8bb01da41d652dc6d, State: Initialized, Role: FOLLOWER
I20260808 18:21:47.322938 25979 raft_consensus.cc:348] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.323155 25979 raft_consensus.cc:374] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:47.323240 25979 raft_consensus.cc:729] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468bb92880cc4fd5bb8a6f62bb0ace3a, State: Initialized, Role: FOLLOWER
I20260808 18:21:47.323333 25983 consensus_queue.cc:260] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.323760 25979 consensus_queue.cc:260] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.323696 25980 raft_consensus.cc:348] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.323974 25980 raft_consensus.cc:374] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:47.324033 25560 heartbeater.cc:503] Master 127.27.18.254:34825 was elected leader, sending a full tablet report...
I20260808 18:21:47.324086 25980 raft_consensus.cc:729] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08d55bcc13c4697a8869a3653e9262f, State: Initialized, Role: FOLLOWER
I20260808 18:21:47.324362 25983 ts_tablet_manager.cc:1428] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 18:21:47.324507 25980 consensus_queue.cc:260] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.324730 25979 ts_tablet_manager.cc:1428] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:21:47.325038 25955 heartbeater.cc:503] Master 127.27.18.254:34825 was elected leader, sending a full tablet report...
I20260808 18:21:47.325147 25753 heartbeater.cc:503] Master 127.27.18.254:34825 was elected leader, sending a full tablet report...
I20260808 18:21:47.325361 25980 ts_tablet_manager.cc:1428] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260808 18:21:47.364609 25561 tablet.cc:2369] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:47.473970 25991 raft_consensus.cc:482] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:47.474215 25991 raft_consensus.cc:504] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.475531 25991 leader_election.cc:288] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bef5cc8849c4cd8bb01da41d652dc6d (127.27.18.193:34131), d08d55bcc13c4697a8869a3653e9262f (127.27.18.194:41931)
I20260808 18:21:47.480264 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.475677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:34131 (local address 127.27.18.195:43021)
0808 18:21:47.475834 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.475849 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.475945 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.476313 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.476318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.476345 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:47.476482 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.476488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.477522 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.477526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.478926 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.478931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.479738 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.479741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.479924 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.479962 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.480039 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.480117 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":169}
I20260808 18:21:47.480273 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.475780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:41931 (local address 127.27.18.195:51373)
0808 18:21:47.476120 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.476131 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:47.476220 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:47.476486 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:47.476489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:47.476504 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:47.476637 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.476642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.477547 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.477550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.478919 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:47.478931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.479738 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.479741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:47.479927 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.479972 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 18:21:47.480073 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 18:21:47.480118 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":152,"thread_start_us":149,"threads_started":1}
I20260808 18:21:47.480791 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.475784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:43021 (local address 127.27.18.193:34131)
0808 18:21:47.475930 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.475937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.475958 (+ 21us) server_negotiation.cc:407] Connection header received
0808 18:21:47.476110 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.476114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.476171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.476285 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:47.476611 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.477396 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.479039 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.479624 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.479737 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.479787 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.480567 (+ 780us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.480660 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:21:47.480792 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.475961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:51373 (local address 127.27.18.194:41931)
0808 18:21:47.476083 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.476087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.476222 (+ 135us) server_negotiation.cc:407] Connection header received
0808 18:21:47.476331 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.476333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.476384 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.476452 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:47.476746 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.477453 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.479039 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.479624 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.479737 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.479787 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.480141 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.480292 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:21:47.481377 25689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61572d839fd34bccaa20c080de1c10c7" candidate_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08d55bcc13c4697a8869a3653e9262f" is_pre_election: true
I20260808 18:21:47.481410 25489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61572d839fd34bccaa20c080de1c10c7" candidate_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" is_pre_election: true
I20260808 18:21:47.481669 25689 raft_consensus.cc:2454] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468bb92880cc4fd5bb8a6f62bb0ace3a in term 0.
I20260808 18:21:47.481683 25489 raft_consensus.cc:2454] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468bb92880cc4fd5bb8a6f62bb0ace3a in term 0.
I20260808 18:21:47.482126 25780 leader_election.cc:302] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468bb92880cc4fd5bb8a6f62bb0ace3a, d08d55bcc13c4697a8869a3653e9262f; no voters:
I20260808 18:21:47.482381 25991 raft_consensus.cc:2781] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:47.482457 25991 raft_consensus.cc:482] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:47.482486 25991 raft_consensus.cc:3037] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:47.483489 25991 raft_consensus.cc:504] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.483835 25991 leader_election.cc:288] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [CANDIDATE]: Term 1 election: Requested vote from peers 3bef5cc8849c4cd8bb01da41d652dc6d (127.27.18.193:34131), d08d55bcc13c4697a8869a3653e9262f (127.27.18.194:41931)
I20260808 18:21:47.484189 25689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61572d839fd34bccaa20c080de1c10c7" candidate_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d08d55bcc13c4697a8869a3653e9262f"
I20260808 18:21:47.484205 25489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61572d839fd34bccaa20c080de1c10c7" candidate_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d"
I20260808 18:21:47.484290 25689 raft_consensus.cc:3037] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:47.484323 25489 raft_consensus.cc:3037] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:47.485280 25689 raft_consensus.cc:2454] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468bb92880cc4fd5bb8a6f62bb0ace3a in term 1.
I20260808 18:21:47.485317 25489 raft_consensus.cc:2454] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468bb92880cc4fd5bb8a6f62bb0ace3a in term 1.
I20260808 18:21:47.485572 25780 leader_election.cc:302] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468bb92880cc4fd5bb8a6f62bb0ace3a, d08d55bcc13c4697a8869a3653e9262f; no voters:
I20260808 18:21:47.485760 25991 raft_consensus.cc:2781] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:47.485919 25991 raft_consensus.cc:686] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 1 LEADER]: Becoming Leader. State: Replica: 468bb92880cc4fd5bb8a6f62bb0ace3a, State: Running, Role: LEADER
I20260808 18:21:47.486219 25991 consensus_queue.cc:237] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } }
I20260808 18:21:47.489100 25269 catalog_manager.cc:5495] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 468bb92880cc4fd5bb8a6f62bb0ace3a (127.27.18.195). New cstate: current_term: 1 leader_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468bb92880cc4fd5bb8a6f62bb0ace3a" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 42043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 } health_report { overall_health: UNKNOWN } } }
W20260808 18:21:47.509351 25956 tablet.cc:2369] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:47.534348 25977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.530962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.27.18.195:42043)
0808 18:21:47.531103 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.531107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.531120 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:21:47.531266 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.531269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.531324 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.531461 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:47.531680 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.532116 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.532957 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.533163 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.533275 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.533576 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:47.533940 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:47.534079 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.534137 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.534191 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:21:47.540041 25489 raft_consensus.cc:1261] T 61572d839fd34bccaa20c080de1c10c7 P 3bef5cc8849c4cd8bb01da41d652dc6d [term 1 FOLLOWER]: Refusing update from remote peer 468bb92880cc4fd5bb8a6f62bb0ace3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:47.540060 25689 raft_consensus.cc:1261] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 1 FOLLOWER]: Refusing update from remote peer 468bb92880cc4fd5bb8a6f62bb0ace3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:47.540764 25995 consensus_queue.cc:1035] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d08d55bcc13c4697a8869a3653e9262f" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 41931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:47.540977 25991 consensus_queue.cc:1035] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bef5cc8849c4cd8bb01da41d652dc6d" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:47.556218 25267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:47.556346 25267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:47.558641 25267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c01a03e43ef645e6a9d6d67070dc4655: Sending DeleteTablet for 3 replicas of tablet 61572d839fd34bccaa20c080de1c10c7
I20260808 18:21:47.559340 25848 tablet_service.cc:1514] Processing DeleteTablet for tablet 61572d839fd34bccaa20c080de1c10c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:47 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:40684
I20260808 18:21:47.559340 25461 tablet_service.cc:1514] Processing DeleteTablet for tablet 61572d839fd34bccaa20c080de1c10c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:47 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:50614
I20260808 18:21:47.559504 25654 tablet_service.cc:1514] Processing DeleteTablet for tablet 61572d839fd34bccaa20c080de1c10c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:47 UTC) from {username='kudu', principal='kudu/127.27.18.254@KRBTEST.COM'} at 127.0.0.1:44324
I20260808 18:21:47.560026 26015 tablet_replica.cc:330] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: stopping tablet replica
I20260808 18:21:47.560025 26012 tablet_replica.cc:330] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: stopping tablet replica
I20260808 18:21:47.560286 26012 raft_consensus.cc:2229] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:21:47.560428 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 25365
I20260808 18:21:47.577684 26015 raft_consensus.cc:2229] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:47.577961 26012 raft_consensus.cc:2258] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:47.578044 26015 raft_consensus.cc:2258] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:47.579551 26012 ts_tablet_manager.cc:1905] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:47.579551 26015 ts_tablet_manager.cc:1905] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:47.582273 26015 ts_tablet_manager.cc:1918] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:47.582273 26012 ts_tablet_manager.cc:1918] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:47.582381 26015 log.cc:1196] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/61572d839fd34bccaa20c080de1c10c7
I20260808 18:21:47.582501 26012 log.cc:1196] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/61572d839fd34bccaa20c080de1c10c7
I20260808 18:21:47.582717 26015 ts_tablet_manager.cc:1939] T 61572d839fd34bccaa20c080de1c10c7 P d08d55bcc13c4697a8869a3653e9262f: Deleting consensus metadata
I20260808 18:21:47.582976 26012 ts_tablet_manager.cc:1939] T 61572d839fd34bccaa20c080de1c10c7 P 468bb92880cc4fd5bb8a6f62bb0ace3a: Deleting consensus metadata
I20260808 18:21:47.584008 25246 catalog_manager.cc:4841] TS 468bb92880cc4fd5bb8a6f62bb0ace3a (127.27.18.195:42043): tablet 61572d839fd34bccaa20c080de1c10c7 (table test-table [id=69c0a00d37bf44f1a9a555af767b469f]) successfully deleted
I20260808 18:21:47.584013 25245 catalog_manager.cc:4841] TS d08d55bcc13c4697a8869a3653e9262f (127.27.18.194:41931): tablet 61572d839fd34bccaa20c080de1c10c7 (table test-table [id=69c0a00d37bf44f1a9a555af767b469f]) successfully deleted
W20260808 18:21:47.585857 25251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:34131 (error 108)
I20260808 18:21:47.586122 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 25567
I20260808 18:21:47.588856 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.588569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:34131 (local address 127.0.0.1:50616)
0808 18:21:47.588703 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:47.588770 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:34131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 18:21:47.589617 25251 catalog_manager.cc:4584] TS 3bef5cc8849c4cd8bb01da41d652dc6d (127.27.18.193:34131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61572d839fd34bccaa20c080de1c10c7: Network error: Client connection negotiation failed: client connection to 127.27.18.193:34131: connect: Connection refused (error 111)
I20260808 18:21:47.596601 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 25759
I20260808 18:21:47.608672 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 25220
2026-08-08T18:21:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2466 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T18:21:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:47Z Disabled control of system clock
I20260808 18:21:47.643545 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38483
--webserver_interface=127.27.18.254
--webserver_port=0
--builtin_ntp_servers=127.27.18.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:38483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:21:47.748500 26023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:47.748703 26023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:47.748749 26023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:47.752369 26023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:47.752413 26023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:47.752429 26023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:47.752444 26023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:47.752458 26023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:47.756664 26023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:38483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:38483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:47.757462 26023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:47.758669 26023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:47.766567 26029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:47.766580 26034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:47.766593 26031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:47.766866 26023 server_base.cc:1034] running on GCE node
I20260808 18:21:47.767583 26023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:47.768549 26023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:47.769740 26023 hybrid_clock.cc:648] HybridClock initialized: now 1786213307769719 us; error 44 us; skew 500 ppm
I20260808 18:21:47.771507 26023 webserver.cc:460] Webserver started at http://127.27.18.254:35163/ using document root <none> and password file <none>
I20260808 18:21:47.772051 26023 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:47.772146 26023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:47.772356 26023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:47.774163 26023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "e4458394e50d905ef9947ee79f126e3f"
server_key_iv: "05de4da5f76fa73a29a5365f48466842"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.774696 26023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "e4458394e50d905ef9947ee79f126e3f"
server_key_iv: "05de4da5f76fa73a29a5365f48466842"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.779120 26023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 18:21:47.782361 26040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:47.784114 26023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:21:47.784257 26023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a"
format_stamp: "Formatted at 2026-08-08 18:21:47 on dist-test-slave-x4qh"
server_key: "e4458394e50d905ef9947ee79f126e3f"
server_key_iv: "05de4da5f76fa73a29a5365f48466842"
server_key_version: "encryptionkey@0"
I20260808 18:21:47.784409 26023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:47.812505 26023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:47.813403 26023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:47.813611 26023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:47.823508 26023 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:38483
I20260808 18:21:47.823509 26125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:38483 every 8 connection(s)
I20260808 18:21:47.824857 26023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:47.828656 26126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:47.829134 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 26023
I20260808 18:21:47.829260 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:47.829639 27723 external_mini_cluster.cc:1428] Setting key ce6fa9becf27ba74d3be54cdb5384415
I20260808 18:21:47.834384 26126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a: Bootstrap starting.
I20260808 18:21:47.836570 26126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:47.837288 26126 log.cc:824] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:47.838999 26126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a: No bootstrap required, opened a new log
I20260808 18:21:47.841897 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:47.831076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.27.18.254:38483)
0808 18:21:47.831756 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:47.831771 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:47.834775 (+ 3004us) server_negotiation.cc:407] Connection header received
0808 18:21:47.835594 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:47.835611 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:47.836061 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:47.836374 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:47.836829 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.837823 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.838670 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:47.839208 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:47.839584 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:47.840333 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:47.840338 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:47.840349 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:47.840691 (+ 342us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:47.840740 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:47.840757 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:47.840864 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:47.841010 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 18:21:47.841257 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":133,"threads_started":1}
I20260808 18:21:47.842552 26126 raft_consensus.cc:348] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38483 } }
I20260808 18:21:47.842782 26126 raft_consensus.cc:374] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:47.842855 26126 raft_consensus.cc:729] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53301f1e8a04bc7b684bcef1d3cfb0a, State: Initialized, Role: FOLLOWER
I20260808 18:21:47.843273 26126 consensus_queue.cc:260] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38483 } }
I20260808 18:21:47.843426 26126 raft_consensus.cc:388] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:47.843488 26126 raft_consensus.cc:482] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:47.843566 26126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:47.844497 26126 raft_consensus.cc:504] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38483 } }
I20260808 18:21:47.844763 26126 leader_election.cc:302] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b53301f1e8a04bc7b684bcef1d3cfb0a; no voters:
I20260808 18:21:47.844990 26126 leader_election.cc:288] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:47.845105 26132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:47.845371 26132 raft_consensus.cc:686] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [term 1 LEADER]: Becoming Leader. State: Replica: b53301f1e8a04bc7b684bcef1d3cfb0a, State: Running, Role: LEADER
I20260808 18:21:47.845646 26132 consensus_queue.cc:237] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38483 } }
I20260808 18:21:47.846155 26126 sys_catalog.cc:564] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:47.847240 26136 sys_catalog.cc:455] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b53301f1e8a04bc7b684bcef1d3cfb0a. Latest consensus state: current_term: 1 leader_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38483 } } }
I20260808 18:21:47.847396 26136 sys_catalog.cc:458] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:47.847714 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53301f1e8a04bc7b684bcef1d3cfb0a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 38483 } } }
I20260808 18:21:47.847818 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:47.847779 26147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:47.850508 26147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:47.855274 26147 catalog_manager.cc:1347] Generated new cluster ID: f3481078429343669156a49701bf66a8
I20260808 18:21:47.855345 26147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:47.865653 26147 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:47.866638 26147 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:47.872948 26147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a: Generated new TSK 0
I20260808 18:21:47.873610 26147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:47.880244 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.193:0
--local_ip_for_outbound_sockets=127.27.18.193
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38483
--builtin_ntp_servers=127.27.18.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 18:21:47.988921 26163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:47.989167 26163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:47.989229 26163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:47.993065 26163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:47.993136 26163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:47.993221 26163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.193
I20260808 18:21:47.997566 26163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.18.193
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.193
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:47.998556 26163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:47.999864 26163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:48.008875 26179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:48.008919 26173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:48.008895 26174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:48.009043 26163 server_base.cc:1034] running on GCE node
I20260808 18:21:48.009500 26163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:48.010088 26163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:48.011271 26163 hybrid_clock.cc:648] HybridClock initialized: now 1786213308011254 us; error 33 us; skew 500 ppm
I20260808 18:21:48.012982 26163 webserver.cc:460] Webserver started at http://127.27.18.193:32939/ using document root <none> and password file <none>
I20260808 18:21:48.013536 26163 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:48.013622 26163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:48.013852 26163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:48.015815 26163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data/instance:
uuid: "53f5c605a731459cab33b2aa747888eb"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "9015e14c160226515c15395c052f8763"
server_key_iv: "b12db6b235912e9175320df431dd68d3"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.016322 26163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance:
uuid: "53f5c605a731459cab33b2aa747888eb"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "9015e14c160226515c15395c052f8763"
server_key_iv: "b12db6b235912e9175320df431dd68d3"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.019747 26163 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:21:48.021891 26187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.022957 26163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:48.023087 26163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal
uuid: "53f5c605a731459cab33b2aa747888eb"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "9015e14c160226515c15395c052f8763"
server_key_iv: "b12db6b235912e9175320df431dd68d3"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.023191 26163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:48.039274 26163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:48.039896 26163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:48.040076 26163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:48.040588 26163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:48.041455 26163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:48.041523 26163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.041590 26163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:48.041633 26163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.050666 26163 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.193:41203
I20260808 18:21:48.050696 26367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.193:41203 every 8 connection(s)
I20260808 18:21:48.051514 26163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/data/info.pb
I20260808 18:21:48.057577 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 26163
I20260808 18:21:48.057742 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-0/wal/instance
I20260808 18:21:48.058053 27723 external_mini_cluster.cc:1428] Setting key ba3fcb663c280c7b763f13762f05ad49
I20260808 18:21:48.060025 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.194:0
--local_ip_for_outbound_sockets=127.27.18.194
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38483
--builtin_ntp_servers=127.27.18.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:21:48.065464 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.053211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.193:44665 (local address 127.27.18.254:38483)
0808 18:21:48.053375 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.053379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.054349 (+ 970us) server_negotiation.cc:407] Connection header received
0808 18:21:48.055761 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.055764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.055809 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.055881 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:48.056721 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.057255 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.058743 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.058993 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.059099 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.064808 (+ 5709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:48.064816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:48.064820 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:48.064837 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:48.064867 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:48.064881 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:48.064974 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.065113 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.065238 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 18:21:48.066228 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.053580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38483 (local address 127.27.18.193:44665)
0808 18:21:48.054200 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.054237 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.055556 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.056039 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.056045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.056129 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:48.056587 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.056598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.057406 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.057412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.058613 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.058624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.059149 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.059153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.059315 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.064540 (+ 5225us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:48.064541 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:48.064577 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:48.064643 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:48.064973 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:48.064980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:48.064984 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.065071 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.065356 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":106,"threads_started":1}
I20260808 18:21:48.067380 26369 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38483
I20260808 18:21:48.067653 26369 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:48.068281 26369 heartbeater.cc:511] Master 127.27.18.254:38483 requested a full tablet report, sending...
I20260808 18:21:48.069775 26071 ts_manager.cc:194] Registered new tserver with Master: 53f5c605a731459cab33b2aa747888eb (127.27.18.193:41203)
I20260808 18:21:48.070870 26071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.193:44665
W20260808 18:21:48.166317 26374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:48.166512 26374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:48.166566 26374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:48.170284 26374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:48.170331 26374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:48.170413 26374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.194
I20260808 18:21:48.175341 26374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.18.194
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.194
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:48.176294 26374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:48.177477 26374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:48.186077 26384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:48.186092 26387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:48.186093 26383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:48.186125 26374 server_base.cc:1034] running on GCE node
I20260808 18:21:48.186733 26374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:48.187266 26374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:48.188432 26374 hybrid_clock.cc:648] HybridClock initialized: now 1786213308188407 us; error 35 us; skew 500 ppm
I20260808 18:21:48.190093 26374 webserver.cc:460] Webserver started at http://127.27.18.194:35533/ using document root <none> and password file <none>
I20260808 18:21:48.190595 26374 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:48.190649 26374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:48.190871 26374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:48.192619 26374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data/instance:
uuid: "4ba6fd3241c24a00941c2f93962bb22b"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "873ef81a43778a413181a8abe24654ce"
server_key_iv: "3d3dc3daa9512fd5bb50efe01a466e7e"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.193120 26374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance:
uuid: "4ba6fd3241c24a00941c2f93962bb22b"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "873ef81a43778a413181a8abe24654ce"
server_key_iv: "3d3dc3daa9512fd5bb50efe01a466e7e"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.196331 26374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:21:48.198429 26396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.199519 26374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:48.199633 26374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal
uuid: "4ba6fd3241c24a00941c2f93962bb22b"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "873ef81a43778a413181a8abe24654ce"
server_key_iv: "3d3dc3daa9512fd5bb50efe01a466e7e"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.199749 26374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:48.213871 26374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:48.214439 26374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:48.214620 26374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:48.215145 26374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:48.216053 26374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:48.216122 26374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.216183 26374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:48.216215 26374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.224433 26374 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.194:34693
I20260808 18:21:48.224476 26560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.194:34693 every 8 connection(s)
I20260808 18:21:48.225201 26374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/data/info.pb
I20260808 18:21:48.226480 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 26374
I20260808 18:21:48.226621 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal/instance
I20260808 18:21:48.226922 27723 external_mini_cluster.cc:1428] Setting key ad14d230695da06b1bab8281c86c7ee4
I20260808 18:21:48.228966 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.18.195:0
--local_ip_for_outbound_sockets=127.27.18.195
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38483
--builtin_ntp_servers=127.27.18.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 18:21:48.234692 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.226809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.194:56029 (local address 127.27.18.254:38483)
0808 18:21:48.226933 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.226937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.228075 (+ 1138us) server_negotiation.cc:407] Connection header received
0808 18:21:48.229737 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.229739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.229781 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.229847 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:48.230860 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.231616 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.232613 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.232976 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.233109 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.234074 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:48.234080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:48.234085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:48.234101 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:48.234143 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:48.234158 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:48.234246 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.234410 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.234483 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:21:48.235325 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.227220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38483 (local address 127.27.18.194:56029)
0808 18:21:48.227923 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.227967 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.229518 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.230014 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.230024 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.230116 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:48.230701 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.230722 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.231732 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.231734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.232474 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.232483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.233137 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.233139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.233278 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.233839 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:48.233841 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:48.233873 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:48.233926 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:48.234251 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:48.234258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:48.234262 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.234351 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.234585 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":184,"threads_started":1}
I20260808 18:21:48.236359 26561 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38483
I20260808 18:21:48.236596 26561 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:48.237150 26561 heartbeater.cc:511] Master 127.27.18.254:38483 requested a full tablet report, sending...
I20260808 18:21:48.238165 26071 ts_manager.cc:194] Registered new tserver with Master: 4ba6fd3241c24a00941c2f93962bb22b (127.27.18.194:34693)
I20260808 18:21:48.238664 26071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.194:56029
W20260808 18:21:48.334228 26567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:48.334436 26567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:48.334471 26567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:48.338230 26567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:48.338295 26567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:48.338372 26567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.18.195
I20260808 18:21:48.342475 26567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.18.195
--webserver_port=0
--tserver_master_addrs=127.27.18.254:38483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.18.195
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:48.343317 26567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:48.344497 26567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:48.353320 26577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:48.353336 26583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:48.353421 26579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:48.353518 26567 server_base.cc:1034] running on GCE node
I20260808 18:21:48.353869 26567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:48.354462 26567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:48.355634 26567 hybrid_clock.cc:648] HybridClock initialized: now 1786213308355624 us; error 22 us; skew 500 ppm
I20260808 18:21:48.357270 26567 webserver.cc:460] Webserver started at http://127.27.18.195:44629/ using document root <none> and password file <none>
I20260808 18:21:48.357761 26567 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:48.357843 26567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:48.358065 26567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:48.359823 26567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data/instance:
uuid: "6ba66cab8c424533afdd535f31f32562"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "4106a0b134bf9630e7ec9526961f23f2"
server_key_iv: "03d9d31d78c3fe4cb2e8e82a939be599"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.360319 26567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance:
uuid: "6ba66cab8c424533afdd535f31f32562"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "4106a0b134bf9630e7ec9526961f23f2"
server_key_iv: "03d9d31d78c3fe4cb2e8e82a939be599"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.363668 26567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:21:48.365893 26591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.366860 26567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:48.366968 26567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal
uuid: "6ba66cab8c424533afdd535f31f32562"
format_stamp: "Formatted at 2026-08-08 18:21:48 on dist-test-slave-x4qh"
server_key: "4106a0b134bf9630e7ec9526961f23f2"
server_key_iv: "03d9d31d78c3fe4cb2e8e82a939be599"
server_key_version: "encryptionkey@0"
I20260808 18:21:48.367077 26567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:48.378183 26567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:48.378680 26567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:48.378847 26567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:48.379354 26567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 18:21:48.380234 26567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 18:21:48.380276 26567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.380334 26567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 18:21:48.380371 26567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:48.388576 26567 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.195:39567
I20260808 18:21:48.388597 26777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.195:39567 every 8 connection(s)
I20260808 18:21:48.389387 26567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/data/info.pb
I20260808 18:21:48.395859 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 26567
I20260808 18:21:48.395979 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal/instance
I20260808 18:21:48.396260 27723 external_mini_cluster.cc:1428] Setting key 6b2c8a9b1e95bc1acdc6bf0cbc3509d8
I20260808 18:21:48.398343 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.391240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:57525 (local address 127.27.18.254:38483)
0808 18:21:48.391392 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.391397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.392356 (+ 959us) server_negotiation.cc:407] Connection header received
0808 18:21:48.393592 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.393594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.393638 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.393705 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:48.394468 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.394965 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.396440 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.396745 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.396840 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.397881 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:48.397885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:48.397887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:48.397899 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:48.397920 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:48.397929 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:48.398016 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.398168 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.398225 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:21:48.399080 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.391642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:38483 (local address 127.27.18.195:57525)
0808 18:21:48.392214 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.392249 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.393417 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.393844 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.393849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.393915 (+ 66us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:21:48.394351 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.394362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.395066 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.395069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.396302 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.396323 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.396884 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.396890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.397087 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.397662 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 18:21:48.397663 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:21:48.397706 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 18:21:48.397756 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:21:48.398024 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:21:48.398031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:21:48.398035 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.398147 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.398402 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":99,"threads_started":1}
I20260808 18:21:48.400018 26778 heartbeater.cc:347] Connected to a master server at 127.27.18.254:38483
I20260808 18:21:48.400221 26778 heartbeater.cc:464] Registering TS with master...
I20260808 18:21:48.400647 26778 heartbeater.cc:511] Master 127.27.18.254:38483 requested a full tablet report, sending...
I20260808 18:21:48.401513 26071 ts_manager.cc:194] Registered new tserver with Master: 6ba66cab8c424533afdd535f31f32562 (127.27.18.195:39567)
I20260808 18:21:48.402009 26071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.18.195:57525
I20260808 18:21:48.409884 27723 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 18:21:48.415107 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.411616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.27.18.254:38483)
0808 18:21:48.411717 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.411722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.411893 (+ 171us) server_negotiation.cc:407] Connection header received
0808 18:21:48.412031 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.412033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.412070 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.412150 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:48.412409 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.412852 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.413677 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.413893 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.413991 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.414698 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:48.414701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:48.414704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:48.414715 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:48.414732 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:48.414747 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:48.414839 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.414934 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.414981 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:21:48.417264 26071 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 18:21:48.419888 26071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 18:21:48.433175 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.427546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.195:39567 (local address 127.0.0.1:37518)
0808 18:21:48.428197 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.428236 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.428365 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.429241 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.429243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.429265 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:48.429418 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.429425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.430726 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.430729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.431827 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.431838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.432702 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.432705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.432863 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.432915 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.432982 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.433055 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":90,"thread_start_us":63,"threads_started":1}
I20260808 18:21:48.433259 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.427640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.27.18.195:39567)
0808 18:21:48.428886 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.428895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.428911 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:21:48.428960 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.428963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.429080 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.429239 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:48.429553 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.430615 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.431947 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.432564 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.432832 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.432980 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.433029 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.433119 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1173,"thread_start_us":121,"threads_started":1}
I20260808 18:21:48.433570 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.427683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:41203 (local address 127.0.0.1:51004)
0808 18:21:48.428560 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.428571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.428655 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.429906 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.429909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.429928 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:48.430003 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.430009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.431217 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.431220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.432187 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.432197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.433171 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.433174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.433271 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.433313 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.433377 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.433441 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":4,"spinlock_wait_cycles":1408,"thread_start_us":87,"threads_started":1}
I20260808 18:21:48.433835 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.427957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:34693 (local address 127.0.0.1:52524)
0808 18:21:48.428575 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.428589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.428656 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.429197 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.429203 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.429224 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:48.429391 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.429398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.430481 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.430484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.431717 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.431727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.432680 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.432683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.433452 (+ 769us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.433471 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.433536 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.433600 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":54,"spinlock_wait_cycles":4096,"thread_start_us":100,"threads_started":1}
I20260808 18:21:48.434084 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.428206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.27.18.193:41203)
0808 18:21:48.429450 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.429463 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.429487 (+ 24us) server_negotiation.cc:407] Connection header received
0808 18:21:48.429560 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.429565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.429731 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.429890 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:48.430132 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.431101 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.432323 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.433056 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.433155 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.433756 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.433800 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.433877 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142,"thread_start_us":126,"threads_started":1}
I20260808 18:21:48.434495 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.428347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.27.18.194:34693)
0808 18:21:48.428819 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.428828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.428847 (+ 19us) server_negotiation.cc:407] Connection header received
0808 18:21:48.428908 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.428912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.429041 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.429149 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:48.429511 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.430365 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.431849 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.432552 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.432677 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.432863 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.434247 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.434350 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":98,"threads_started":1}
I20260808 18:21:48.436209 26677 tablet_service.cc:1467] Processing CreateTablet for tablet 16f83289f1704c319a87fe69d62dc585 (DEFAULT_TABLE table=test-table [id=57b3b6889a6a4b82bbc8614ea697fbd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:48.436509 26266 tablet_service.cc:1467] Processing CreateTablet for tablet 16f83289f1704c319a87fe69d62dc585 (DEFAULT_TABLE table=test-table [id=57b3b6889a6a4b82bbc8614ea697fbd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:48.436599 26461 tablet_service.cc:1467] Processing CreateTablet for tablet 16f83289f1704c319a87fe69d62dc585 (DEFAULT_TABLE table=test-table [id=57b3b6889a6a4b82bbc8614ea697fbd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 18:21:48.437211 26677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16f83289f1704c319a87fe69d62dc585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:48.437350 26266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16f83289f1704c319a87fe69d62dc585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:48.437533 26461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16f83289f1704c319a87fe69d62dc585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:48.443104 26816 tablet_bootstrap.cc:492] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Bootstrap starting.
I20260808 18:21:48.444129 26819 tablet_bootstrap.cc:492] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Bootstrap starting.
I20260808 18:21:48.445183 26816 tablet_bootstrap.cc:654] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:48.445833 26819 tablet_bootstrap.cc:654] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:48.445843 26816 log.cc:824] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:48.446543 26819 log.cc:824] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:48.446547 26818 tablet_bootstrap.cc:492] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb: Bootstrap starting.
I20260808 18:21:48.447897 26816 tablet_bootstrap.cc:492] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: No bootstrap required, opened a new log
I20260808 18:21:48.448076 26816 ts_tablet_manager.cc:1397] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 18:21:48.448413 26819 tablet_bootstrap.cc:492] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: No bootstrap required, opened a new log
I20260808 18:21:48.448621 26819 ts_tablet_manager.cc:1397] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:48.448850 26818 tablet_bootstrap.cc:654] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:48.449484 26818 log.cc:824] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:48.450994 26818 tablet_bootstrap.cc:492] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb: No bootstrap required, opened a new log
I20260808 18:21:48.451195 26818 ts_tablet_manager.cc:1397] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 18:21:48.452289 26816 raft_consensus.cc:348] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.452404 26819 raft_consensus.cc:348] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.452543 26816 raft_consensus.cc:374] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:48.452576 26819 raft_consensus.cc:374] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:48.452600 26816 raft_consensus.cc:729] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba6fd3241c24a00941c2f93962bb22b, State: Initialized, Role: FOLLOWER
I20260808 18:21:48.452618 26819 raft_consensus.cc:729] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba66cab8c424533afdd535f31f32562, State: Initialized, Role: FOLLOWER
I20260808 18:21:48.453001 26816 consensus_queue.cc:260] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.453001 26819 consensus_queue.cc:260] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.453897 26819 ts_tablet_manager.cc:1428] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 18:21:48.453897 26816 ts_tablet_manager.cc:1428] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 18:21:48.454232 26561 heartbeater.cc:503] Master 127.27.18.254:38483 was elected leader, sending a full tablet report...
I20260808 18:21:48.454245 26778 heartbeater.cc:503] Master 127.27.18.254:38483 was elected leader, sending a full tablet report...
I20260808 18:21:48.454365 26818 raft_consensus.cc:348] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.454584 26818 raft_consensus.cc:374] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:48.454632 26818 raft_consensus.cc:729] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f5c605a731459cab33b2aa747888eb, State: Initialized, Role: FOLLOWER
I20260808 18:21:48.455065 26818 consensus_queue.cc:260] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.455727 26369 heartbeater.cc:503] Master 127.27.18.254:38483 was elected leader, sending a full tablet report...
I20260808 18:21:48.456640 26818 ts_tablet_manager.cc:1428] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
W20260808 18:21:48.475996 26562 tablet.cc:2369] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:21:48.478644 26825 raft_consensus.cc:482] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 18:21:48.478816 26825 raft_consensus.cc:504] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.479703 26825 leader_election.cc:288] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53f5c605a731459cab33b2aa747888eb (127.27.18.193:41203), 4ba6fd3241c24a00941c2f93962bb22b (127.27.18.194:34693)
I20260808 18:21:48.484112 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.479959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:42611 (local address 127.27.18.193:41203)
0808 18:21:48.480086 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.480090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.480107 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:21:48.480247 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.480250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.480284 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.480346 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:48.480662 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.481388 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.482841 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.483421 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.483562 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.483608 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.483929 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.483995 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:21:48.484107 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.479990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.18.195:37551 (local address 127.27.18.194:34693)
0808 18:21:48.480111 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.480115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.480411 (+ 296us) server_negotiation.cc:407] Connection header received
0808 18:21:48.480478 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.480480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.480518 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.480575 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 18:21:48.480825 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.481506 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.482838 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.483436 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.483562 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.483608 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.483930 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.483995 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:21:48.484124 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.479879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:41203 (local address 127.27.18.195:42611)
0808 18:21:48.480054 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.480071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.480149 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.480388 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.480402 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.480414 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:48.480532 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.480538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.481488 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.481490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.482705 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.482716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.483566 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.483570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.483756 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.483817 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.483897 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.483978 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":111}
I20260808 18:21:48.484143 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.479918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.194:34693 (local address 127.27.18.195:37551)
0808 18:21:48.480282 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.480292 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:21:48.480408 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:21:48.480617 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:21:48.480619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:21:48.480639 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 18:21:48.480725 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.480729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.481596 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.481599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.482709 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:21:48.482716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.483566 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.483570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:21:48.483758 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.483817 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 18:21:48.483897 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 18:21:48.483978 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":93,"thread_start_us":110,"threads_started":1}
I20260808 18:21:48.484714 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16f83289f1704c319a87fe69d62dc585" candidate_uuid: "6ba66cab8c424533afdd535f31f32562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f5c605a731459cab33b2aa747888eb" is_pre_election: true
I20260808 18:21:48.484741 26493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16f83289f1704c319a87fe69d62dc585" candidate_uuid: "6ba66cab8c424533afdd535f31f32562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba6fd3241c24a00941c2f93962bb22b" is_pre_election: true
I20260808 18:21:48.484954 26298 raft_consensus.cc:2454] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba66cab8c424533afdd535f31f32562 in term 0.
I20260808 18:21:48.484977 26493 raft_consensus.cc:2454] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba66cab8c424533afdd535f31f32562 in term 0.
I20260808 18:21:48.485426 26598 leader_election.cc:302] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f5c605a731459cab33b2aa747888eb, 6ba66cab8c424533afdd535f31f32562; no voters:
I20260808 18:21:48.485787 26825 raft_consensus.cc:2781] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 18:21:48.485865 26825 raft_consensus.cc:482] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 18:21:48.485895 26825 raft_consensus.cc:3037] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:48.486796 26825 raft_consensus.cc:504] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.487100 26825 leader_election.cc:288] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [CANDIDATE]: Term 1 election: Requested vote from peers 53f5c605a731459cab33b2aa747888eb (127.27.18.193:41203), 4ba6fd3241c24a00941c2f93962bb22b (127.27.18.194:34693)
I20260808 18:21:48.487453 26493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16f83289f1704c319a87fe69d62dc585" candidate_uuid: "6ba66cab8c424533afdd535f31f32562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba6fd3241c24a00941c2f93962bb22b"
I20260808 18:21:48.487455 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16f83289f1704c319a87fe69d62dc585" candidate_uuid: "6ba66cab8c424533afdd535f31f32562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f5c605a731459cab33b2aa747888eb"
I20260808 18:21:48.487581 26493 raft_consensus.cc:3037] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:48.487607 26298 raft_consensus.cc:3037] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:48.488768 26298 raft_consensus.cc:2454] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba66cab8c424533afdd535f31f32562 in term 1.
I20260808 18:21:48.488768 26493 raft_consensus.cc:2454] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba66cab8c424533afdd535f31f32562 in term 1.
I20260808 18:21:48.489087 26600 leader_election.cc:302] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ba6fd3241c24a00941c2f93962bb22b, 53f5c605a731459cab33b2aa747888eb, 6ba66cab8c424533afdd535f31f32562; no voters:
I20260808 18:21:48.489287 26825 raft_consensus.cc:2781] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:48.489475 26825 raft_consensus.cc:686] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 1 LEADER]: Becoming Leader. State: Replica: 6ba66cab8c424533afdd535f31f32562, State: Running, Role: LEADER
I20260808 18:21:48.489727 26825 consensus_queue.cc:237] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } }
I20260808 18:21:48.492311 26068 catalog_manager.cc:5495] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ba66cab8c424533afdd535f31f32562 (127.27.18.195). New cstate: current_term: 1 leader_uuid: "6ba66cab8c424533afdd535f31f32562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba66cab8c424533afdd535f31f32562" member_type: VOTER last_known_addr { host: "127.27.18.195" port: 39567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20260808 18:21:48.508347 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.505391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.27.18.195:39567)
0808 18:21:48.505510 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:48.505516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:48.505563 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:21:48.505736 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:48.505739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:48.505790 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:48.505867 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:21:48.506061 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.506458 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.507173 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:48.507353 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:48.507471 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:48.507689 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:21:48.507999 (+ 310us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:21:48.508077 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:48.508154 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 18:21:48.508190 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 18:21:48.513135 26493 raft_consensus.cc:1261] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 1 FOLLOWER]: Refusing update from remote peer 6ba66cab8c424533afdd535f31f32562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:48.513135 26298 raft_consensus.cc:1261] T 16f83289f1704c319a87fe69d62dc585 P 53f5c605a731459cab33b2aa747888eb [term 1 FOLLOWER]: Refusing update from remote peer 6ba66cab8c424533afdd535f31f32562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:21:48.513715 26830 consensus_queue.cc:1035] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53f5c605a731459cab33b2aa747888eb" member_type: VOTER last_known_addr { host: "127.27.18.193" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:48.513875 26825 consensus_queue.cc:1035] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba6fd3241c24a00941c2f93962bb22b" member_type: VOTER last_known_addr { host: "127.27.18.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:21:48.520102 26836 mvcc.cc:205] Tried to move back new op lower bound from 7316329711663902720 to 7316329711576301568. Current Snapshot: MvccSnapshot[applied={T|T < 7316329711663902720}]
I20260808 18:21:48.527735 26071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:48.527871 26071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 18:21:48.529841 26071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b53301f1e8a04bc7b684bcef1d3cfb0a: Sending DeleteTablet for 3 replicas of tablet 16f83289f1704c319a87fe69d62dc585
I20260808 18:21:48.530467 26266 tablet_service.cc:1514] Processing DeleteTablet for tablet 16f83289f1704c319a87fe69d62dc585 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:48 UTC) from {username='slave'} at 127.0.0.1:51004
I20260808 18:21:48.530493 26677 tablet_service.cc:1514] Processing DeleteTablet for tablet 16f83289f1704c319a87fe69d62dc585 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:48 UTC) from {username='slave'} at 127.0.0.1:37518
I20260808 18:21:48.531070 26858 tablet_replica.cc:330] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: stopping tablet replica
I20260808 18:21:48.531162 26461 tablet_service.cc:1514] Processing DeleteTablet for tablet 16f83289f1704c319a87fe69d62dc585 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 18:21:48 UTC) from {username='slave'} at 127.0.0.1:52524
I20260808 18:21:48.531337 26858 raft_consensus.cc:2229] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 1 LEADER]: Raft consensus shutting down.
I20260808 18:21:48.531534 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 26163
I20260808 18:21:48.536916 26859 tablet_replica.cc:330] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: stopping tablet replica
I20260808 18:21:48.537338 26859 raft_consensus.cc:2229] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 18:21:48.537724 26859 raft_consensus.cc:2258] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:48.539892 26859 ts_tablet_manager.cc:1905] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:48.542030 26858 raft_consensus.cc:2258] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 18:21:48.542943 26859 ts_tablet_manager.cc:1918] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:48.543049 26859 log.cc:1196] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-1/wal/wals/16f83289f1704c319a87fe69d62dc585
I20260808 18:21:48.543259 26858 ts_tablet_manager.cc:1905] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 18:21:48.543310 26859 ts_tablet_manager.cc:1939] T 16f83289f1704c319a87fe69d62dc585 P 4ba6fd3241c24a00941c2f93962bb22b: Deleting consensus metadata
I20260808 18:21:48.544353 26047 catalog_manager.cc:4841] TS 4ba6fd3241c24a00941c2f93962bb22b (127.27.18.194:34693): tablet 16f83289f1704c319a87fe69d62dc585 (table test-table [id=57b3b6889a6a4b82bbc8614ea697fbd8]) successfully deleted
I20260808 18:21:48.545760 26858 ts_tablet_manager.cc:1918] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 18:21:48.545878 26858 log.cc:1196] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Deleting WAL directory at /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786213058405439-27723-0/minicluster-data/ts-2/wal/wals/16f83289f1704c319a87fe69d62dc585
I20260808 18:21:48.546185 26858 ts_tablet_manager.cc:1939] T 16f83289f1704c319a87fe69d62dc585 P 6ba66cab8c424533afdd535f31f32562: Deleting consensus metadata
I20260808 18:21:48.547178 26048 catalog_manager.cc:4841] TS 6ba66cab8c424533afdd535f31f32562 (127.27.18.195:39567): tablet 16f83289f1704c319a87fe69d62dc585 (table test-table [id=57b3b6889a6a4b82bbc8614ea697fbd8]) successfully deleted
W20260808 18:21:48.554386 26042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.18.193:41203 (error 108)
I20260808 18:21:48.555199 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 26374
I20260808 18:21:48.561301 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:48.555052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.193:41203 (local address 127.0.0.1:51020)
0808 18:21:48.555260 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 18:21:48.555335 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.193:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260808 18:21:48.561501 26042 catalog_manager.cc:4584] TS 53f5c605a731459cab33b2aa747888eb (127.27.18.193:41203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16f83289f1704c319a87fe69d62dc585: Network error: Client connection negotiation failed: client connection to 127.27.18.193:41203: connect: Connection refused (error 111)
I20260808 18:21:48.566324 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 26567
I20260808 18:21:48.574451 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 26023
2026-08-08T18:21:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (964 ms)
[----------] 6 tests from EncryptionPolicyTest (10487 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:48 dist-test-slave-x4qh krb5kdc[26863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:48 dist-test-slave-x4qh krb5kdc[26863](info): set up 2 sockets
Aug 08 18:21:48 dist-test-slave-x4qh krb5kdc[26863](info): commencing operation
krb5kdc: starting...
W20260808 18:21:49.737675 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:49 dist-test-slave-x4qh krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:49.850188 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:49.953630 26880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:49.953835 26880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:49.953904 26880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:49.957558 26880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:49.957621 26880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:49.957640 26880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:49.957656 26880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:49.957670 26880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:49.962469 26880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:49.963375 26880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:49.964257 26880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:49.972048 26887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:49.972076 26886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:49.972105 26890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:49.972177 26880 server_base.cc:1034] running on GCE node
I20260808 18:21:49.972723 26880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:49.973650 26880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:49.974809 26880 hybrid_clock.cc:648] HybridClock initialized: now 1786213309974789 us; error 27 us; skew 500 ppm
Aug 08 18:21:49 dist-test-slave-x4qh krb5kdc[26863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213309, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:49.977757 26880 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:49.978606 26880 webserver.cc:460] Webserver started at http://127.0.0.1:33039/ using document root <none> and password file <none>
I20260808 18:21:49.979126 26880 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:49.979218 26880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:49.979489 26880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:49.981319 26880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "4c523defa6d64632ae11039ec810ae56"
format_stamp: "Formatted at 2026-08-08 18:21:49 on dist-test-slave-x4qh"
server_key: "b5742900b58deed8c7ba7647292fdb14"
server_key_iv: "964100f1e531c744d36f535759c50bd3"
server_key_version: "encryptionkey@0"
I20260808 18:21:49.982065 26880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "4c523defa6d64632ae11039ec810ae56"
format_stamp: "Formatted at 2026-08-08 18:21:49 on dist-test-slave-x4qh"
server_key: "b5742900b58deed8c7ba7647292fdb14"
server_key_iv: "964100f1e531c744d36f535759c50bd3"
server_key_version: "encryptionkey@0"
I20260808 18:21:49.985737 26880 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:21:49.987844 26900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:49.988950 26880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:49.989087 26880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4c523defa6d64632ae11039ec810ae56"
format_stamp: "Formatted at 2026-08-08 18:21:49 on dist-test-slave-x4qh"
server_key: "b5742900b58deed8c7ba7647292fdb14"
server_key_iv: "964100f1e531c744d36f535759c50bd3"
server_key_version: "encryptionkey@0"
I20260808 18:21:49.989202 26880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:50.008023 26880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:50.010802 26880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:50.011000 26880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:50.017731 26880 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46243
I20260808 18:21:50.017757 26988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46243 every 8 connection(s)
I20260808 18:21:50.018538 26880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:50.020895 26990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:50.025704 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 26880
I20260808 18:21:50.025846 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:50.025913 26990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56: Bootstrap starting.
I20260808 18:21:50.026140 27723 external_mini_cluster.cc:1428] Setting key 9f5e032a9fa7c4f2ed905c6d0305f13e
I20260808 18:21:50.028185 26990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:50.028838 26990 log.cc:824] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:50.030441 26990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56: No bootstrap required, opened a new log
I20260808 18:21:50.033720 26990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c523defa6d64632ae11039ec810ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46243 } }
Aug 08 18:21:50 dist-test-slave-x4qh krb5kdc[26863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:21:50.033974 26990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:50.034034 26990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c523defa6d64632ae11039ec810ae56, State: Initialized, Role: FOLLOWER
I20260808 18:21:50.034487 26990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c523defa6d64632ae11039ec810ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46243 } }
I20260808 18:21:50.034649 26990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:50.034736 26990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:50.034858 26990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:50.035964 26990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c523defa6d64632ae11039ec810ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46243 } }
I20260808 18:21:50.036289 26990 leader_election.cc:302] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c523defa6d64632ae11039ec810ae56; no voters:
I20260808 18:21:50.036630 26990 leader_election.cc:288] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:50.036733 26998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:50.036963 26998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [term 1 LEADER]: Becoming Leader. State: Replica: 4c523defa6d64632ae11039ec810ae56, State: Running, Role: LEADER
I20260808 18:21:50.037338 26998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c523defa6d64632ae11039ec810ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46243 } }
I20260808 18:21:50.037869 26990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:50.039717 26999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c523defa6d64632ae11039ec810ae56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c523defa6d64632ae11039ec810ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46243 } } }
I20260808 18:21:50.039830 26999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:50.040221 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:50.027354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51959 (local address 127.0.0.1:46243)
0808 18:21:50.027726 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:50.027738 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:50.027767 (+ 29us) server_negotiation.cc:407] Connection header received
0808 18:21:50.028392 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:50.028408 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:50.028688 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:50.028977 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:50.029733 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:50.030668 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:50.031491 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:50.031910 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:50.032066 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:50.034664 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:50.034670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:50.034686 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:50.034716 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:50.036622 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:50.037328 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:50.037333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:50.037339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:50.037477 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:50.037846 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:50.037849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:50.037851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:50.038302 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:50.038503 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:50.038979 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 18:21:50.039207 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
I20260808 18:21:50.040375 27000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c523defa6d64632ae11039ec810ae56. Latest consensus state: current_term: 1 leader_uuid: "4c523defa6d64632ae11039ec810ae56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c523defa6d64632ae11039ec810ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46243 } } }
I20260808 18:21:50.040591 27000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56 [sys.catalog]: This master's current role is: LEADER
W20260808 18:21:50.042375 27017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:21:50.042615 27017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:21:50.042706 27020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:50.043419 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:50.043672 27020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:50.049198 27020 catalog_manager.cc:1347] Generated new cluster ID: b19db33bf7bf4266aa3d66273a43a3dc
I20260808 18:21:50.049536 27020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:50.053264 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:50.045578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44905 (local address 127.0.0.1:46243)
0808 18:21:50.045658 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:50.045661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:50.045865 (+ 204us) server_negotiation.cc:407] Connection header received
0808 18:21:50.045954 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:50.045957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:50.045994 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:50.046068 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:50.046795 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:50.047536 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:50.048480 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:50.048765 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:50.048846 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:50.050226 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:50.050230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:50.050232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:50.050253 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:50.051677 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:50.052130 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:50.052135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:50.052140 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:50.052202 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:50.052495 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:50.052498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:50.052499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:50.052640 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:50.052785 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:50.053052 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 18:21:50.053134 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 18:21:50.088567 27020 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:50.089475 27020 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:50.094681 27020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c523defa6d64632ae11039ec810ae56: Generated new TSK 0
I20260808 18:21:50.095302 27020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:50.100922 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 26880
W20260808 18:21:50.112624 26861 connection.cc:535] client connection to 127.0.0.1:46243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46243): Connection reset by peer (error 104)
2026-08-08T18:21:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:50 dist-test-slave-x4qh krb5kdc[27039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:50 dist-test-slave-x4qh krb5kdc[27039](info): set up 2 sockets
Aug 08 18:21:50 dist-test-slave-x4qh krb5kdc[27039](info): commencing operation
krb5kdc: starting...
W20260808 18:21:51.269500 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:51 dist-test-slave-x4qh krb5kdc[27039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:51.382368 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:51.484853 27060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:51.485059 27060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:51.485097 27060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:51.488586 27060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:51.488631 27060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:51.488651 27060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:51.488665 27060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:51.488679 27060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:51.493225 27060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:51.494015 27060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:51.494849 27060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:51.502482 27065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:51.502516 27067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:51.502651 27060 server_base.cc:1034] running on GCE node
W20260808 18:21:51.502516 27070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:51.503230 27060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:51.504205 27060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:51.505363 27060 hybrid_clock.cc:648] HybridClock initialized: now 1786213311505346 us; error 28 us; skew 500 ppm
Aug 08 18:21:51 dist-test-slave-x4qh krb5kdc[27039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:51.508260 27060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:51.509346 27060 webserver.cc:460] Webserver started at http://127.0.0.1:44991/ using document root <none> and password file <none>
I20260808 18:21:51.509985 27060 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:51.510068 27060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:51.510288 27060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:51.512676 27060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "7ec0c963ff37418caca33a76aa061f79"
format_stamp: "Formatted at 2026-08-08 18:21:51 on dist-test-slave-x4qh"
server_key: "01b8ba172dc5c3d6c4b444a64fabba05"
server_key_iv: "5a20075e84688f4f4d8dabc7817b0a72"
server_key_version: "encryptionkey@0"
I20260808 18:21:51.513319 27060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "7ec0c963ff37418caca33a76aa061f79"
format_stamp: "Formatted at 2026-08-08 18:21:51 on dist-test-slave-x4qh"
server_key: "01b8ba172dc5c3d6c4b444a64fabba05"
server_key_iv: "5a20075e84688f4f4d8dabc7817b0a72"
server_key_version: "encryptionkey@0"
I20260808 18:21:51.517417 27060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 18:21:51.519539 27082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:51.520589 27060 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:51.520679 27060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "7ec0c963ff37418caca33a76aa061f79"
format_stamp: "Formatted at 2026-08-08 18:21:51 on dist-test-slave-x4qh"
server_key: "01b8ba172dc5c3d6c4b444a64fabba05"
server_key_iv: "5a20075e84688f4f4d8dabc7817b0a72"
server_key_version: "encryptionkey@0"
I20260808 18:21:51.520763 27060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:51.535920 27060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:51.538667 27060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:51.538795 27060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:51.545406 27060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42999
I20260808 18:21:51.545463 27171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42999 every 8 connection(s)
I20260808 18:21:51.546227 27060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:51.548044 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27060
I20260808 18:21:51.548195 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:51.548471 27723 external_mini_cluster.cc:1428] Setting key 2b92903d07efe9fcee9e6e8c6581902f
I20260808 18:21:51.549005 27173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:51.555506 27173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79: Bootstrap starting.
I20260808 18:21:51.557706 27179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:51.549756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44739 (local address 127.0.0.1:42999)
0808 18:21:51.550185 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:51.550202 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:51.550239 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:21:51.551016 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:51.551038 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:51.551458 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:51.551798 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:51.552187 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:51.553504 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:51.554294 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:51.554977 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:51.555123 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:51.555923 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:51.555929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:51.555943 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:51.556372 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:51.556449 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:51.556477 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:51.556624 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:51.556740 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 18:21:51.556866 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":115,"threads_started":1}
I20260808 18:21:51.558092 27173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:51.558890 27173 log.cc:824] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:51.560618 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:51.560925 27173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79: No bootstrap required, opened a new log
I20260808 18:21:51.564899 27173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec0c963ff37418caca33a76aa061f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42999 } }
I20260808 18:21:51.565125 27173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:51.565197 27173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec0c963ff37418caca33a76aa061f79, State: Initialized, Role: FOLLOWER
I20260808 18:21:51.565636 27173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec0c963ff37418caca33a76aa061f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42999 } }
I20260808 18:21:51.565809 27173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:51.565884 27173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:51.565989 27173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:51.566602 27179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:51.562603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53225 (local address 127.0.0.1:42999)
0808 18:21:51.562702 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:51.562710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:51.562882 (+ 172us) server_negotiation.cc:407] Connection header received
0808 18:21:51.563009 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:51.563011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:51.563043 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:51.563104 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:51.563434 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:51.563874 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:51.564938 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:51.565205 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:51.565306 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:51.566111 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:51.566115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:51.566117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:51.566130 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:51.566159 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:51.566169 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:51.566239 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:51.566424 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 18:21:51.566480 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:21:51.567117 27173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec0c963ff37418caca33a76aa061f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42999 } }
I20260808 18:21:51.567492 27173 leader_election.cc:302] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ec0c963ff37418caca33a76aa061f79; no voters:
I20260808 18:21:51.567816 27173 leader_election.cc:288] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:51.567871 27189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:51.568040 27189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec0c963ff37418caca33a76aa061f79, State: Running, Role: LEADER
I20260808 18:21:51.568315 27189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec0c963ff37418caca33a76aa061f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42999 } }
I20260808 18:21:51.568948 27173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:51.569818 27191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ec0c963ff37418caca33a76aa061f79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec0c963ff37418caca33a76aa061f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42999 } } }
I20260808 18:21:51.569890 27190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ec0c963ff37418caca33a76aa061f79. Latest consensus state: current_term: 1 leader_uuid: "7ec0c963ff37418caca33a76aa061f79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec0c963ff37418caca33a76aa061f79" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42999 } } }
I20260808 18:21:51.570030 27190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:51.570029 27191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:51.570468 27203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:51.573587 27203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:51.578840 27203 catalog_manager.cc:1347] Generated new cluster ID: 03bd00b6fcd6404684c52964feb85c12
I20260808 18:21:51.578919 27203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:51.593451 27203 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:51.594467 27203 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:51.599258 27203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ec0c963ff37418caca33a76aa061f79: Generated new TSK 0
I20260808 18:21:51.599855 27203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:51.603861 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27060
W20260808 18:21:51.621204 27036 connection.cc:535] client connection to 127.0.0.1:42999 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42999): Connection reset by peer (error 104)
2026-08-08T18:21:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:51 dist-test-slave-x4qh krb5kdc[27222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:51 dist-test-slave-x4qh krb5kdc[27222](info): set up 2 sockets
Aug 08 18:21:51 dist-test-slave-x4qh krb5kdc[27222](info): commencing operation
krb5kdc: starting...
W20260808 18:21:52.774085 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:52 dist-test-slave-x4qh krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:52.888213 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:52.989059 27245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:52.989290 27245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:52.989362 27245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:52.992853 27245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:52.992934 27245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:52.992971 27245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:52.993031 27245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:52.993067 27245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:52.997774 27245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:52.998582 27245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:52.999418 27245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:53.007055 27255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:53.007023 27257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:53.007014 27254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:53.007288 27245 server_base.cc:1034] running on GCE node
I20260808 18:21:53.007825 27245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:53.008667 27245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:53.009819 27245 hybrid_clock.cc:648] HybridClock initialized: now 1786213313009793 us; error 37 us; skew 500 ppm
Aug 08 18:21:53 dist-test-slave-x4qh krb5kdc[27222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:53.012104 27245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:53.012775 27245 webserver.cc:460] Webserver started at http://127.0.0.1:35289/ using document root <none> and password file <none>
I20260808 18:21:53.013195 27245 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:53.013247 27245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:53.013393 27245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:53.015127 27245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "ef04bb6776c447419e862c97fd86ae0e"
format_stamp: "Formatted at 2026-08-08 18:21:53 on dist-test-slave-x4qh"
server_key: "70647a800a0af301bd14814c353b5350"
server_key_iv: "481d198595b00a92ec28dc58cff0583f"
server_key_version: "encryptionkey@0"
I20260808 18:21:53.015684 27245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "ef04bb6776c447419e862c97fd86ae0e"
format_stamp: "Formatted at 2026-08-08 18:21:53 on dist-test-slave-x4qh"
server_key: "70647a800a0af301bd14814c353b5350"
server_key_iv: "481d198595b00a92ec28dc58cff0583f"
server_key_version: "encryptionkey@0"
I20260808 18:21:53.018680 27245 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 18:21:53.020685 27266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:53.021706 27245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 18:21:53.021823 27245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "ef04bb6776c447419e862c97fd86ae0e"
format_stamp: "Formatted at 2026-08-08 18:21:53 on dist-test-slave-x4qh"
server_key: "70647a800a0af301bd14814c353b5350"
server_key_iv: "481d198595b00a92ec28dc58cff0583f"
server_key_version: "encryptionkey@0"
I20260808 18:21:53.021924 27245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:53.025805 27245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:53.028194 27245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:53.028371 27245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:53.034291 27245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46619
I20260808 18:21:53.034309 27355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46619 every 8 connection(s)
I20260808 18:21:53.035142 27245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:53.037783 27357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:53.042517 27357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e: Bootstrap starting.
I20260808 18:21:53.043462 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27245
I20260808 18:21:53.043628 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:53.043941 27723 external_mini_cluster.cc:1428] Setting key 5a4e50aa2020d92b973eab661f11797a
I20260808 18:21:53.045322 27357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:53.045989 27357 log.cc:824] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:53.048033 27357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e: No bootstrap required, opened a new log
I20260808 18:21:53.048422 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:53.044955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51503 (local address 127.0.0.1:46619)
0808 18:21:53.045339 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:53.045352 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:53.045409 (+ 57us) server_negotiation.cc:407] Connection header received
0808 18:21:53.045967 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:53.045981 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:53.046252 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:53.046585 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:53.047281 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:53.047285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:53.047295 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:53.047311 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:53.047353 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:53.047411 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:53.047505 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:53.047629 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 18:21:53.047768 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":124,"threads_started":1}
I20260808 18:21:53.051044 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:53.051921 27357 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef04bb6776c447419e862c97fd86ae0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46619 } }
I20260808 18:21:53.052189 27357 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:53.052249 27357 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef04bb6776c447419e862c97fd86ae0e, State: Initialized, Role: FOLLOWER
I20260808 18:21:53.052711 27357 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef04bb6776c447419e862c97fd86ae0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46619 } }
I20260808 18:21:53.052866 27357 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:53.052937 27357 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:53.053053 27357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:53.054013 27357 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef04bb6776c447419e862c97fd86ae0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46619 } }
I20260808 18:21:53.054359 27357 leader_election.cc:302] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef04bb6776c447419e862c97fd86ae0e; no voters:
I20260808 18:21:53.054606 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:53.053197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59107 (local address 127.0.0.1:46619)
0808 18:21:53.053329 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:53.053333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:53.053517 (+ 184us) server_negotiation.cc:407] Connection header received
0808 18:21:53.053586 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:53.053589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:53.053618 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:53.053684 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:53.054280 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:53.054283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:53.054285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:53.054295 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:53.054318 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:53.054328 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:53.054413 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:53.054450 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 18:21:53.054483 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 18:21:53.054751 27357 leader_election.cc:288] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:53.055023 27374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:53.055573 27374 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [term 1 LEADER]: Becoming Leader. State: Replica: ef04bb6776c447419e862c97fd86ae0e, State: Running, Role: LEADER
I20260808 18:21:53.055925 27374 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef04bb6776c447419e862c97fd86ae0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46619 } }
I20260808 18:21:53.056082 27357 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:53.057421 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef04bb6776c447419e862c97fd86ae0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef04bb6776c447419e862c97fd86ae0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46619 } } }
I20260808 18:21:53.057539 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:53.057765 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef04bb6776c447419e862c97fd86ae0e. Latest consensus state: current_term: 1 leader_uuid: "ef04bb6776c447419e862c97fd86ae0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef04bb6776c447419e862c97fd86ae0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46619 } } }
I20260808 18:21:53.057842 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e [sys.catalog]: This master's current role is: LEADER
W20260808 18:21:53.060351 27401 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:21:53.060437 27401 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:21:53.060509 27393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:53.061257 27393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:53.065618 27393 catalog_manager.cc:1347] Generated new cluster ID: da3e8aa4c49842d18ac9f000b65ba16a
I20260808 18:21:53.065687 27393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:53.080598 27393 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:53.080884 27393 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:53.089102 27393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef04bb6776c447419e862c97fd86ae0e: Generated new TSK 0
I20260808 18:21:53.089740 27393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:53.103785 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27245
2026-08-08T18:21:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:53 dist-test-slave-x4qh krb5kdc[27408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:53 dist-test-slave-x4qh krb5kdc[27408](info): set up 2 sockets
Aug 08 18:21:53 dist-test-slave-x4qh krb5kdc[27408](info): commencing operation
krb5kdc: starting...
W20260808 18:21:54.260548 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:54 dist-test-slave-x4qh krb5kdc[27408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:54.372097 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:54.471113 27430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:54.471300 27430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:54.471333 27430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:54.474892 27430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:54.474942 27430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:54.474961 27430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:54.474977 27430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:54.474990 27430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:54.479745 27430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:54.480533 27430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:54.481360 27430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:54.488992 27446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:54.488991 27445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:54.488991 27448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:54.489058 27430 server_base.cc:1034] running on GCE node
I20260808 18:21:54.489704 27430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:54.490650 27430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:54.491835 27430 hybrid_clock.cc:648] HybridClock initialized: now 1786213314491818 us; error 32 us; skew 500 ppm
Aug 08 18:21:54 dist-test-slave-x4qh krb5kdc[27408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:54.494349 27430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:54.495157 27430 webserver.cc:460] Webserver started at http://127.0.0.1:34581/ using document root <none> and password file <none>
I20260808 18:21:54.495723 27430 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:54.495828 27430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:54.496052 27430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:54.497881 27430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "3c3c5643c48443108e71e682db80a64c"
format_stamp: "Formatted at 2026-08-08 18:21:54 on dist-test-slave-x4qh"
server_key: "5857afbe7319aa3f0180fc4909317771"
server_key_iv: "46a7773a189bc2d6c66d12277cfd77b5"
server_key_version: "encryptionkey@0"
I20260808 18:21:54.498430 27430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "3c3c5643c48443108e71e682db80a64c"
format_stamp: "Formatted at 2026-08-08 18:21:54 on dist-test-slave-x4qh"
server_key: "5857afbe7319aa3f0180fc4909317771"
server_key_iv: "46a7773a189bc2d6c66d12277cfd77b5"
server_key_version: "encryptionkey@0"
I20260808 18:21:54.501901 27430 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 18:21:54.504096 27463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:54.505226 27430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:54.505367 27430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "3c3c5643c48443108e71e682db80a64c"
format_stamp: "Formatted at 2026-08-08 18:21:54 on dist-test-slave-x4qh"
server_key: "5857afbe7319aa3f0180fc4909317771"
server_key_iv: "46a7773a189bc2d6c66d12277cfd77b5"
server_key_version: "encryptionkey@0"
I20260808 18:21:54.505472 27430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:54.526299 27430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:54.528841 27430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:54.529031 27430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:54.535458 27430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38717
I20260808 18:21:54.535485 27546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38717 every 8 connection(s)
I20260808 18:21:54.536300 27430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:54.537176 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27430
I20260808 18:21:54.537334 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:54.537586 27723 external_mini_cluster.cc:1428] Setting key 727d8594593380152baad663231b5d5b
I20260808 18:21:54.539119 27548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:54.545717 27548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c: Bootstrap starting.
Aug 08 18:21:54 dist-test-slave-x4qh krb5kdc[27408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:21:54.548302 27548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:54.549130 27548 log.cc:824] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:54.551249 27548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c: No bootstrap required, opened a new log
I20260808 18:21:54.551733 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:54.538911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41677 (local address 127.0.0.1:38717)
0808 18:21:54.539350 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:54.539365 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:54.539440 (+ 75us) server_negotiation.cc:407] Connection header received
0808 18:21:54.540033 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:54.540048 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:54.540346 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:54.540596 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:54.541500 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:54.542418 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:54.543544 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:54.543973 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:54.544081 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:54.546869 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:54.546876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:54.546892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:54.546926 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:54.548834 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:54.549233 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:54.549239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:54.549245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:54.549375 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:54.549691 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:54.549693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:54.549694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:54.550155 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:54.550337 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:54.550573 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 18:21:54.550869 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260808 18:21:54.554317 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:54.555287 27548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3c5643c48443108e71e682db80a64c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 18:21:54.555576 27548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:54.555637 27548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3c5643c48443108e71e682db80a64c, State: Initialized, Role: FOLLOWER
I20260808 18:21:54.556079 27548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3c5643c48443108e71e682db80a64c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 18:21:54.556219 27548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:54.556286 27548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:54.556427 27548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:54.557233 27548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3c5643c48443108e71e682db80a64c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 18:21:54.557562 27548 leader_election.cc:302] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c3c5643c48443108e71e682db80a64c; no voters:
I20260808 18:21:54.557919 27548 leader_election.cc:288] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:54.557967 27563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:54.558221 27563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [term 1 LEADER]: Becoming Leader. State: Replica: 3c3c5643c48443108e71e682db80a64c, State: Running, Role: LEADER
I20260808 18:21:54.558555 27563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3c5643c48443108e71e682db80a64c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 18:21:54.559150 27548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:54.559818 27563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c3c5643c48443108e71e682db80a64c. Latest consensus state: current_term: 1 leader_uuid: "3c3c5643c48443108e71e682db80a64c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3c5643c48443108e71e682db80a64c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } } }
I20260808 18:21:54.559964 27563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:54.560257 27564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c3c5643c48443108e71e682db80a64c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c3c5643c48443108e71e682db80a64c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } } }
I20260808 18:21:54.560384 27564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c [sys.catalog]: This master's current role is: LEADER
W20260808 18:21:54.564052 27589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:21:54.564189 27589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:21:54.564287 27571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:54.564990 27571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:54.566844 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:54.556254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51727 (local address 127.0.0.1:38717)
0808 18:21:54.556394 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:54.556398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:54.556534 (+ 136us) server_negotiation.cc:407] Connection header received
0808 18:21:54.556656 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:54.556658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:54.556702 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:54.556781 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:54.557377 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:54.557943 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:54.559047 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:54.559417 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:54.559514 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:54.561156 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:54.561159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:54.561162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:54.561202 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:54.562940 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:54.563439 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:54.563442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:54.563444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:54.563523 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:54.563832 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:54.563835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:54.563837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:54.564031 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:54.564157 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:54.566580 (+ 2423us) server_negotiation.cc:299] Negotiation successful
0808 18:21:54.566686 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:21:54.570171 27571 catalog_manager.cc:1347] Generated new cluster ID: 1821bc16872d42428258664bf0edc0ab
I20260808 18:21:54.570232 27571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:54.601848 27571 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:54.603060 27571 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:54.616472 27571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c3c5643c48443108e71e682db80a64c: Generated new TSK 0
I20260808 18:21:54.617251 27571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:54.625200 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27430
2026-08-08T18:21:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:54 dist-test-slave-x4qh krb5kdc[27599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:54 dist-test-slave-x4qh krb5kdc[27599](info): set up 2 sockets
Aug 08 18:21:54 dist-test-slave-x4qh krb5kdc[27599](info): commencing operation
krb5kdc: starting...
W20260808 18:21:55.782904 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:55 dist-test-slave-x4qh krb5kdc[27599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:55.897466 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:55.998646 27617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:55.998899 27617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:55.998965 27617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:56.002417 27617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:56.002499 27617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:56.002535 27617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:56.002588 27617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:56.002625 27617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:56.007223 27617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:56.008149 27617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:56.009109 27617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:56.017095 27629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:56.017207 27617 server_base.cc:1034] running on GCE node
W20260808 18:21:56.017118 27626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:56.017099 27633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:56.017910 27617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:56.018978 27617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:56.020157 27617 hybrid_clock.cc:648] HybridClock initialized: now 1786213316020128 us; error 42 us; skew 500 ppm
Aug 08 18:21:56 dist-test-slave-x4qh krb5kdc[27599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213316, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:56.022683 27617 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:56.023613 27617 webserver.cc:460] Webserver started at http://127.0.0.1:32875/ using document root <none> and password file <none>
I20260808 18:21:56.024140 27617 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:56.024230 27617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:56.024474 27617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:56.026286 27617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "e88c50f5425c4329b6f6a18908923e30"
format_stamp: "Formatted at 2026-08-08 18:21:56 on dist-test-slave-x4qh"
server_key: "3656759b37cc6ff5b29ed98d6ebdc4b3"
server_key_iv: "1b101544314f33457e604ca687fe0c71"
server_key_version: "encryptionkey@0"
I20260808 18:21:56.026794 27617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "e88c50f5425c4329b6f6a18908923e30"
format_stamp: "Formatted at 2026-08-08 18:21:56 on dist-test-slave-x4qh"
server_key: "3656759b37cc6ff5b29ed98d6ebdc4b3"
server_key_iv: "1b101544314f33457e604ca687fe0c71"
server_key_version: "encryptionkey@0"
I20260808 18:21:56.029978 27617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 18:21:56.032202 27646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:56.033200 27617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:56.033349 27617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "e88c50f5425c4329b6f6a18908923e30"
format_stamp: "Formatted at 2026-08-08 18:21:56 on dist-test-slave-x4qh"
server_key: "3656759b37cc6ff5b29ed98d6ebdc4b3"
server_key_iv: "1b101544314f33457e604ca687fe0c71"
server_key_version: "encryptionkey@0"
I20260808 18:21:56.033447 27617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:56.054082 27617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:56.056470 27617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:56.056676 27617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:56.063218 27617 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38075
I20260808 18:21:56.063248 27731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38075 every 8 connection(s)
I20260808 18:21:56.064070 27617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:56.066480 27732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:56.071115 27732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30: Bootstrap starting.
I20260808 18:21:56.072719 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27617
I20260808 18:21:56.072875 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:56.073141 27723 external_mini_cluster.cc:1428] Setting key 1c7c5fb11de645df98b4f3a74497ee99
I20260808 18:21:56.073388 27732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:56.074128 27732 log.cc:824] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:56.075884 27732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30: No bootstrap required, opened a new log
I20260808 18:21:56.079356 27732 raft_consensus.cc:348] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e88c50f5425c4329b6f6a18908923e30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260808 18:21:56.079594 27732 raft_consensus.cc:374] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:56.079677 27732 raft_consensus.cc:729] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e88c50f5425c4329b6f6a18908923e30, State: Initialized, Role: FOLLOWER
I20260808 18:21:56.080050 27732 consensus_queue.cc:260] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e88c50f5425c4329b6f6a18908923e30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260808 18:21:56.080191 27732 raft_consensus.cc:388] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:56.080255 27732 raft_consensus.cc:482] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:56.080348 27732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:56.081103 27732 raft_consensus.cc:504] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e88c50f5425c4329b6f6a18908923e30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260808 18:21:56.081197 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:56.074127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46587 (local address 127.0.0.1:38075)
0808 18:21:56.074536 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:56.074553 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:56.074592 (+ 39us) server_negotiation.cc:407] Connection header received
0808 18:21:56.075241 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:56.075258 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:56.075608 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:56.075908 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:56.076297 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:56.077148 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:56.077960 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:56.078630 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:56.078768 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:56.079540 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:56.079546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:56.079564 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:56.080023 (+ 459us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:56.080069 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:56.080091 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:56.080205 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:56.080306 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 18:21:56.080578 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":121,"threads_started":1}
I20260808 18:21:56.081442 27732 leader_election.cc:302] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e88c50f5425c4329b6f6a18908923e30; no voters:
I20260808 18:21:56.081733 27732 leader_election.cc:288] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:56.081826 27741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:56.082037 27741 raft_consensus.cc:686] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [term 1 LEADER]: Becoming Leader. State: Replica: e88c50f5425c4329b6f6a18908923e30, State: Running, Role: LEADER
I20260808 18:21:56.082350 27741 consensus_queue.cc:237] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e88c50f5425c4329b6f6a18908923e30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } }
I20260808 18:21:56.082774 27732 sys_catalog.cc:564] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:56.083745 27747 sys_catalog.cc:455] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e88c50f5425c4329b6f6a18908923e30. Latest consensus state: current_term: 1 leader_uuid: "e88c50f5425c4329b6f6a18908923e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e88c50f5425c4329b6f6a18908923e30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } } }
I20260808 18:21:56.083869 27747 sys_catalog.cc:458] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:56.084052 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:56.084234 27742 sys_catalog.cc:455] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e88c50f5425c4329b6f6a18908923e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e88c50f5425c4329b6f6a18908923e30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38075 } } }
I20260808 18:21:56.084317 27742 sys_catalog.cc:458] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:56.085170 27761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:56.088140 27761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:56.090222 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:56.086318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40781 (local address 127.0.0.1:38075)
0808 18:21:56.086412 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:56.086417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:56.086583 (+ 166us) server_negotiation.cc:407] Connection header received
0808 18:21:56.086669 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:56.086671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:56.086709 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:56.086783 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:56.087061 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:56.087731 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:56.088707 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:56.088986 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:56.089084 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:56.089839 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:56.089842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:56.089844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:56.089858 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:56.089881 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:56.089892 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:56.089963 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:56.090066 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 18:21:56.090119 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:21:56.093453 27761 catalog_manager.cc:1347] Generated new cluster ID: 11e31382ed924d0592d96ecf6f659bcb
I20260808 18:21:56.093538 27761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:56.108042 27761 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:56.109098 27761 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:56.119751 27761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e88c50f5425c4329b6f6a18908923e30: Generated new TSK 0
I20260808 18:21:56.120350 27761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:56.127513 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27617
2026-08-08T18:21:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:56 dist-test-slave-x4qh krb5kdc[27792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:56 dist-test-slave-x4qh krb5kdc[27792](info): set up 2 sockets
Aug 08 18:21:56 dist-test-slave-x4qh krb5kdc[27792](info): commencing operation
krb5kdc: starting...
W20260808 18:21:57.288889 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:57 dist-test-slave-x4qh krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:57.402365 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:57.503731 27816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:57.503957 27816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:57.504057 27816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:57.508158 27816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:57.508229 27816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:57.508262 27816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:57.508294 27816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:57.508327 27816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:57.514631 27816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:57.515612 27816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:57.516538 27816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:21:57.524413 27826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:57.524461 27816 server_base.cc:1034] running on GCE node
W20260808 18:21:57.524480 27824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:57.524520 27828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:57.525137 27816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:57.526108 27816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:57.527300 27816 hybrid_clock.cc:648] HybridClock initialized: now 1786213317527285 us; error 32 us; skew 500 ppm
Aug 08 18:21:57 dist-test-slave-x4qh krb5kdc[27792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213317, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:57.529740 27816 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:57.530474 27816 webserver.cc:460] Webserver started at http://127.0.0.1:33839/ using document root <none> and password file <none>
I20260808 18:21:57.530985 27816 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:57.531069 27816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:57.531275 27816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:57.533043 27816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "8c2c7693023b4bba956f56188d8b5733"
format_stamp: "Formatted at 2026-08-08 18:21:57 on dist-test-slave-x4qh"
server_key: "9bd1f996c3f895db7eb037c1b1a96ba7"
server_key_iv: "ac4afb8e1530d87d1099a09f8d548229"
server_key_version: "encryptionkey@0"
I20260808 18:21:57.533536 27816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "8c2c7693023b4bba956f56188d8b5733"
format_stamp: "Formatted at 2026-08-08 18:21:57 on dist-test-slave-x4qh"
server_key: "9bd1f996c3f895db7eb037c1b1a96ba7"
server_key_iv: "ac4afb8e1530d87d1099a09f8d548229"
server_key_version: "encryptionkey@0"
I20260808 18:21:57.536902 27816 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:21:57.538887 27842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:57.540012 27816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:21:57.540150 27816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "8c2c7693023b4bba956f56188d8b5733"
format_stamp: "Formatted at 2026-08-08 18:21:57 on dist-test-slave-x4qh"
server_key: "9bd1f996c3f895db7eb037c1b1a96ba7"
server_key_iv: "ac4afb8e1530d87d1099a09f8d548229"
server_key_version: "encryptionkey@0"
I20260808 18:21:57.540248 27816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:57.544265 27816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:57.547199 27816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:57.547401 27816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:57.553809 27816 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34225
I20260808 18:21:57.553822 27918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34225 every 8 connection(s)
I20260808 18:21:57.554579 27816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:57.557140 27919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:57.557302 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27816
I20260808 18:21:57.557461 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:57.557729 27723 external_mini_cluster.cc:1428] Setting key b1fbd3bce9d2bff1549a1deb9b83418d
I20260808 18:21:57.562245 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:57.558927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36107 (local address 127.0.0.1:34225)
0808 18:21:57.559322 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:57.559335 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:57.559399 (+ 64us) server_negotiation.cc:407] Connection header received
0808 18:21:57.560049 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:57.560065 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:57.560290 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:57.560579 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:57.561226 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:57.561235 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:57.561246 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:57.561256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:57.561310 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:57.561329 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:57.561471 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:57.561540 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 18:21:57.561673 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":103,"threads_started":1}
I20260808 18:21:57.563198 27919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733: Bootstrap starting.
I20260808 18:21:57.564884 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:57.565621 27919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:57.566365 27919 log.cc:824] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:57.568414 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:57.566995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35387 (local address 127.0.0.1:34225)
0808 18:21:57.567118 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:57.567121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:57.567274 (+ 153us) server_negotiation.cc:407] Connection header received
0808 18:21:57.567336 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:57.567338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:57.567383 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:57.567461 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:57.568077 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:57.568080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:57.568082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:21:57.568094 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:57.568118 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:21:57.568130 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:57.568220 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:57.568281 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:21:57.568319 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:21:57.568441 27919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733: No bootstrap required, opened a new log
I20260808 18:21:57.571619 27919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2c7693023b4bba956f56188d8b5733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260808 18:21:57.571841 27919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:57.571919 27919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2c7693023b4bba956f56188d8b5733, State: Initialized, Role: FOLLOWER
I20260808 18:21:57.572295 27919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2c7693023b4bba956f56188d8b5733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260808 18:21:57.572448 27919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:57.572530 27919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:57.572646 27919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:57.573434 27919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2c7693023b4bba956f56188d8b5733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260808 18:21:57.573748 27919 leader_election.cc:302] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c2c7693023b4bba956f56188d8b5733; no voters:
I20260808 18:21:57.574045 27919 leader_election.cc:288] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:57.574127 27939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:57.574345 27939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [term 1 LEADER]: Becoming Leader. State: Replica: 8c2c7693023b4bba956f56188d8b5733, State: Running, Role: LEADER
I20260808 18:21:57.574616 27939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2c7693023b4bba956f56188d8b5733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } }
I20260808 18:21:57.575156 27919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:57.576383 27940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c2c7693023b4bba956f56188d8b5733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2c7693023b4bba956f56188d8b5733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } } }
I20260808 18:21:57.576401 27941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c2c7693023b4bba956f56188d8b5733. Latest consensus state: current_term: 1 leader_uuid: "8c2c7693023b4bba956f56188d8b5733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2c7693023b4bba956f56188d8b5733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34225 } } }
I20260808 18:21:57.576514 27940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:57.576517 27941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:57.577651 27961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:57.580567 27961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:57.585608 27961 catalog_manager.cc:1347] Generated new cluster ID: 977409623d0440c5b64c24ac576232e2
I20260808 18:21:57.585675 27961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:57.604161 27961 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:57.604532 27961 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:57.624855 27961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c2c7693023b4bba956f56188d8b5733: Generated new TSK 0
I20260808 18:21:57.625485 27961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:57.633191 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27816
2026-08-08T18:21:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:57 dist-test-slave-x4qh krb5kdc[27976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:57 dist-test-slave-x4qh krb5kdc[27976](info): set up 2 sockets
Aug 08 18:21:57 dist-test-slave-x4qh krb5kdc[27976](info): commencing operation
krb5kdc: starting...
W20260808 18:21:58.789017 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:21:58 dist-test-slave-x4qh krb5kdc[27976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:21:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:21:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:21:58.905498 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42327
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:21:59.005594 27999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:21:59.005837 27999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:21:59.005909 27999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:21:59.009624 27999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:21:59.009703 27999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:21:59.009742 27999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:21:59.009796 27999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:21:59.009826 27999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:21:59.014407 27999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:21:59.015302 27999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:21:59.016229 27999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:21:59.024006 27999 server_base.cc:1034] running on GCE node
W20260808 18:21:59.024042 28012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:59.023981 28015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:21:59.023988 28010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:21:59.024765 27999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:21:59.025694 27999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:21:59.026857 27999 hybrid_clock.cc:648] HybridClock initialized: now 1786213319026832 us; error 35 us; skew 500 ppm
Aug 08 18:21:59 dist-test-slave-x4qh krb5kdc[27976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213319, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:21:59.032763 27999 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:21:59.033608 27999 webserver.cc:460] Webserver started at http://127.0.0.1:34327/ using document root <none> and password file <none>
I20260808 18:21:59.034132 27999 fs_manager.cc:359] Metadata directory not provided
I20260808 18:21:59.034189 27999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:21:59.034413 27999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:21:59.036339 27999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "1690cfcede3e426180779e3b4fc95ad1"
format_stamp: "Formatted at 2026-08-08 18:21:59 on dist-test-slave-x4qh"
server_key: "adaf0d265c2a6033fdf2e63b661ed872"
server_key_iv: "71207172e3e09b5bbcb30230e5cb17d7"
server_key_version: "encryptionkey@0"
I20260808 18:21:59.036842 27999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "1690cfcede3e426180779e3b4fc95ad1"
format_stamp: "Formatted at 2026-08-08 18:21:59 on dist-test-slave-x4qh"
server_key: "adaf0d265c2a6033fdf2e63b661ed872"
server_key_iv: "71207172e3e09b5bbcb30230e5cb17d7"
server_key_version: "encryptionkey@0"
I20260808 18:21:59.040180 27999 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:21:59.042313 28023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:21:59.043470 27999 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:21:59.043601 27999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "1690cfcede3e426180779e3b4fc95ad1"
format_stamp: "Formatted at 2026-08-08 18:21:59 on dist-test-slave-x4qh"
server_key: "adaf0d265c2a6033fdf2e63b661ed872"
server_key_iv: "71207172e3e09b5bbcb30230e5cb17d7"
server_key_version: "encryptionkey@0"
I20260808 18:21:59.043717 27999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:21:59.060425 27999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:21:59.063066 27999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:21:59.063273 27999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:21:59.070566 27999 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42327
I20260808 18:21:59.070573 28103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42327 every 8 connection(s)
I20260808 18:21:59.071486 27999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:21:59.073861 28104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:21:59.078619 28104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1: Bootstrap starting.
I20260808 18:21:59.080701 28104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1: Neither blocks nor log segments found. Creating new log.
I20260808 18:21:59.081295 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 27999
I20260808 18:21:59.081285 28104 log.cc:824] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1: Log is configured to *not* fsync() on all Append() calls
I20260808 18:21:59.081452 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:21:59.081789 27723 external_mini_cluster.cc:1428] Setting key 8785270c76004a19d7d8cc114c34f258
I20260808 18:21:59.083314 28104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1: No bootstrap required, opened a new log
I20260808 18:21:59.086931 28104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1690cfcede3e426180779e3b4fc95ad1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42327 } }
I20260808 18:21:59.087141 28104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:21:59.087182 28104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1690cfcede3e426180779e3b4fc95ad1, State: Initialized, Role: FOLLOWER
I20260808 18:21:59.087596 28104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1690cfcede3e426180779e3b4fc95ad1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42327 } }
I20260808 18:21:59.087747 28104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:21:59.087817 28104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:21:59.087932 28104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:21:59.088973 28104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1690cfcede3e426180779e3b4fc95ad1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42327 } }
I20260808 18:21:59.089375 28104 leader_election.cc:302] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1690cfcede3e426180779e3b4fc95ad1; no voters:
I20260808 18:21:59.089706 28104 leader_election.cc:288] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:21:59.089874 28112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:21:59.090191 28112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [term 1 LEADER]: Becoming Leader. State: Replica: 1690cfcede3e426180779e3b4fc95ad1, State: Running, Role: LEADER
Aug 08 18:21:59 dist-test-slave-x4qh krb5kdc[27976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:21:59.090478 28112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1690cfcede3e426180779e3b4fc95ad1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42327 } }
I20260808 18:21:59.090848 28104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:21:59.092530 28116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1690cfcede3e426180779e3b4fc95ad1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1690cfcede3e426180779e3b4fc95ad1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42327 } } }
I20260808 18:21:59.092636 28116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:59.092947 28135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:21:59.093261 28117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1690cfcede3e426180779e3b4fc95ad1. Latest consensus state: current_term: 1 leader_uuid: "1690cfcede3e426180779e3b4fc95ad1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1690cfcede3e426180779e3b4fc95ad1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42327 } } }
I20260808 18:21:59.093963 28117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1 [sys.catalog]: This master's current role is: LEADER
I20260808 18:21:59.096136 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:59.082864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:33047 (local address 127.0.0.1:42327)
0808 18:21:59.083237 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:59.083252 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:59.083283 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:21:59.083934 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:59.083949 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:59.084236 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:59.084493 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:59.085395 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:59.086294 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:59.087323 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:59.087883 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:59.088072 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:59.091014 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:59.091020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:59.091034 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:59.091061 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:59.093001 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:59.093540 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:59.093544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:59.093548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:59.093648 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:59.094115 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:59.094119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:59.094121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:59.094558 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:59.094762 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:59.095046 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 18:21:59.095263 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":123,"threads_started":1}
I20260808 18:21:59.096827 28135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:21:59.099031 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:21:59.102460 28135 catalog_manager.cc:1347] Generated new cluster ID: e224de62a12f42adb4ad7955a72bd5d4
I20260808 18:21:59.102548 28135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:21:59.108130 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:21:59.101181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:35661 (local address 127.0.0.1:42327)
0808 18:21:59.101247 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 18:21:59.101250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:21:59.101261 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:21:59.101294 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:21:59.101296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:21:59.101325 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:21:59.101380 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:21:59.102099 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:59.102760 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:59.103544 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:21:59.103770 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:21:59.103871 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:21:59.105038 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:21:59.105041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:21:59.105044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:21:59.105065 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:21:59.106440 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:59.106894 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:59.106899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:59.106905 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:59.106968 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:21:59.107278 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:21:59.107281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:21:59.107283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:21:59.107490 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:21:59.107643 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 18:21:59.107899 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 18:21:59.107978 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 18:21:59.130223 28135 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:21:59.131006 28135 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:21:59.136180 28135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1690cfcede3e426180779e3b4fc95ad1: Generated new TSK 0
I20260808 18:21:59.136739 28135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:21:59.141956 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 27999
W20260808 18:21:59.155512 27973 connection.cc:535] client connection to 127.0.0.1:42327 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42327): Connection reset by peer (error 104)
2026-08-08T18:21:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:21:59 dist-test-slave-x4qh krb5kdc[28155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:21:59 dist-test-slave-x4qh krb5kdc[28155](info): set up 2 sockets
Aug 08 18:21:59 dist-test-slave-x4qh krb5kdc[28155](info): commencing operation
krb5kdc: starting...
W20260808 18:22:00.313385 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:00 dist-test-slave-x4qh krb5kdc[28155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:22:00.424467 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:00.527275 28178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:00.527549 28178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:00.527606 28178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:00.531018 28178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:00.531059 28178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:00.531076 28178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:00.531090 28178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:00.531104 28178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:00.535581 28178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:00.536352 28178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:00.537169 28178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:00.544525 28187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:00.544566 28191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:00.544629 28178 server_base.cc:1034] running on GCE node
W20260808 18:22:00.544577 28188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:00.545240 28178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:00.546142 28178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:00.547308 28178 hybrid_clock.cc:648] HybridClock initialized: now 1786213320547284 us; error 38 us; skew 500 ppm
Aug 08 18:22:00 dist-test-slave-x4qh krb5kdc[28155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213320, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:00.549774 28178 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:22:00.550573 28178 webserver.cc:460] Webserver started at http://127.0.0.1:33111/ using document root <none> and password file <none>
I20260808 18:22:00.551080 28178 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:00.551167 28178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:00.551433 28178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:00.553217 28178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67"
format_stamp: "Formatted at 2026-08-08 18:22:00 on dist-test-slave-x4qh"
server_key: "d84c024ddcd213648964a0bc6fa90514"
server_key_iv: "09a2e61db439c6afe6a13641a3b38b07"
server_key_version: "encryptionkey@0"
I20260808 18:22:00.553733 28178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67"
format_stamp: "Formatted at 2026-08-08 18:22:00 on dist-test-slave-x4qh"
server_key: "d84c024ddcd213648964a0bc6fa90514"
server_key_iv: "09a2e61db439c6afe6a13641a3b38b07"
server_key_version: "encryptionkey@0"
I20260808 18:22:00.556989 28178 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:22:00.559087 28199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:00.560117 28178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:00.560247 28178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67"
format_stamp: "Formatted at 2026-08-08 18:22:00 on dist-test-slave-x4qh"
server_key: "d84c024ddcd213648964a0bc6fa90514"
server_key_iv: "09a2e61db439c6afe6a13641a3b38b07"
server_key_version: "encryptionkey@0"
I20260808 18:22:00.560360 28178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:00.590176 28178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:00.592725 28178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:00.592937 28178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:00.599284 28178 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42777
I20260808 18:22:00.599342 28274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42777 every 8 connection(s)
I20260808 18:22:00.600137 28178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:00.600427 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28178
I20260808 18:22:00.600561 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:00.600875 27723 external_mini_cluster.cc:1428] Setting key f2662867f6f8394ea34e8a9645832f3e
I20260808 18:22:00.602934 28275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:00.608181 28275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67: Bootstrap starting.
I20260808 18:22:00.609200 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:00.602151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:43435 (local address 127.0.0.1:42777)
0808 18:22:00.602581 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:00.602594 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:00.602624 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:22:00.603257 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:00.603275 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:00.603641 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:00.603944 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:00.604298 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:00.605227 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:00.606164 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:00.606624 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:00.606787 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:00.607668 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:00.607672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:00.607682 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:00.608004 (+ 322us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:00.608050 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:00.608069 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:00.608225 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:00.608462 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 18:22:00.608579 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":142,"threads_started":1}
I20260808 18:22:00.610401 28275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:00.611243 28275 log.cc:824] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:00.611879 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:00.613027 28275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67: No bootstrap required, opened a new log
I20260808 18:22:00.616493 28275 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 18:22:00.616737 28275 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:00.616840 28275 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9ecb9ca612435d9cdc8e3bb91eff67, State: Initialized, Role: FOLLOWER
I20260808 18:22:00.617274 28275 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 18:22:00.617427 28275 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:00.617518 28275 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:00.617625 28275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:00.617873 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:00.613674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:39049 (local address 127.0.0.1:42777)
0808 18:22:00.613789 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:00.613794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:00.614009 (+ 215us) server_negotiation.cc:407] Connection header received
0808 18:22:00.614132 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:00.614135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:00.614175 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:00.614260 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:00.614517 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:00.614990 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:00.615859 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:00.616229 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:00.616338 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:00.617344 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:00.617347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:00.617350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:00.617361 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:00.617391 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:00.617401 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:00.617510 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:00.617685 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 18:22:00.617738 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:00.618607 28275 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 18:22:00.618932 28275 leader_election.cc:302] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9ecb9ca612435d9cdc8e3bb91eff67; no voters:
I20260808 18:22:00.619169 28275 leader_election.cc:288] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:00.619305 28288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:00.619544 28288 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [term 1 LEADER]: Becoming Leader. State: Replica: dd9ecb9ca612435d9cdc8e3bb91eff67, State: Running, Role: LEADER
I20260808 18:22:00.619872 28288 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } }
I20260808 18:22:00.620251 28275 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:00.621467 28293 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9ecb9ca612435d9cdc8e3bb91eff67. Latest consensus state: current_term: 1 leader_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } } }
I20260808 18:22:00.621512 28290 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9ecb9ca612435d9cdc8e3bb91eff67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42777 } } }
I20260808 18:22:00.621585 28293 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:00.621605 28290 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:00.622525 28306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:00.625000 28306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:00.629832 28306 catalog_manager.cc:1347] Generated new cluster ID: 850d9b2465a944b2bdfc8bf25d1a79d8
I20260808 18:22:00.629887 28306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:00.635730 28306 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:00.636727 28306 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:00.642306 28306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd9ecb9ca612435d9cdc8e3bb91eff67: Generated new TSK 0
I20260808 18:22:00.642895 28306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:00.650540 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28178
W20260808 18:22:00.666217 28153 connection.cc:535] client connection to 127.0.0.1:42777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42777): Connection reset by peer (error 104)
2026-08-08T18:22:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:00 dist-test-slave-x4qh krb5kdc[28322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:00 dist-test-slave-x4qh krb5kdc[28322](info): set up 2 sockets
Aug 08 18:22:00 dist-test-slave-x4qh krb5kdc[28322](info): commencing operation
krb5kdc: starting...
W20260808 18:22:01.813846 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:01 dist-test-slave-x4qh krb5kdc[28322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:22:01.926146 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46471
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:02.025362 28341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:02.025566 28341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:02.025604 28341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:02.029209 28341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:02.029246 28341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:02.029263 28341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:02.029294 28341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:02.029309 28341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:02.033900 28341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:02.034626 28341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:02.035434 28341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:02.043179 28353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:02.043188 28354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:02.043180 28358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:02.043226 28341 server_base.cc:1034] running on GCE node
I20260808 18:22:02.043975 28341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:02.045013 28341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:02.046193 28341 hybrid_clock.cc:648] HybridClock initialized: now 1786213322046176 us; error 27 us; skew 500 ppm
Aug 08 18:22:02 dist-test-slave-x4qh krb5kdc[28322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:02.048725 28341 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:22:02.049492 28341 webserver.cc:460] Webserver started at http://127.0.0.1:36835/ using document root <none> and password file <none>
I20260808 18:22:02.049993 28341 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:02.050077 28341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:02.050287 28341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:02.052168 28341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee"
format_stamp: "Formatted at 2026-08-08 18:22:02 on dist-test-slave-x4qh"
server_key: "cb9e178c941fb6a10bbca1c253c24069"
server_key_iv: "39866ce06fb0f098d8501359af7eaf2f"
server_key_version: "encryptionkey@0"
I20260808 18:22:02.052687 28341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee"
format_stamp: "Formatted at 2026-08-08 18:22:02 on dist-test-slave-x4qh"
server_key: "cb9e178c941fb6a10bbca1c253c24069"
server_key_iv: "39866ce06fb0f098d8501359af7eaf2f"
server_key_version: "encryptionkey@0"
I20260808 18:22:02.055917 28341 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 18:22:02.058001 28368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:02.059026 28341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:02.059161 28341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee"
format_stamp: "Formatted at 2026-08-08 18:22:02 on dist-test-slave-x4qh"
server_key: "cb9e178c941fb6a10bbca1c253c24069"
server_key_iv: "39866ce06fb0f098d8501359af7eaf2f"
server_key_version: "encryptionkey@0"
I20260808 18:22:02.059265 28341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:02.063313 28341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:02.066179 28341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:02.066361 28341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:02.072695 28341 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46471
I20260808 18:22:02.072706 28451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46471 every 8 connection(s)
I20260808 18:22:02.073478 28341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:02.075896 28453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:02.080682 28453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee: Bootstrap starting.
I20260808 18:22:02.081490 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28341
I20260808 18:22:02.081624 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:02.081884 27723 external_mini_cluster.cc:1428] Setting key e1b43da6be359c8b21968be879e86a43
I20260808 18:22:02.083140 28453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:02.083848 28453 log.cc:824] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:02.085655 28453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee: No bootstrap required, opened a new log
I20260808 18:22:02.086501 28460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:02.082800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:57647 (local address 127.0.0.1:46471)
0808 18:22:02.083270 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:02.083293 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:02.083328 (+ 35us) server_negotiation.cc:407] Connection header received
0808 18:22:02.084059 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:02.084076 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:02.084336 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:02.084643 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:02.085358 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:02.085363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:02.085373 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:02.085383 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:02.085429 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:02.085446 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:02.085556 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:02.085676 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 18:22:02.085839 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
I20260808 18:22:02.089073 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:02.089504 28453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260808 18:22:02.089789 28453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:02.089849 28453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d33ef6f55da4f95b2bbf5c33ab0f2ee, State: Initialized, Role: FOLLOWER
I20260808 18:22:02.090292 28453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260808 18:22:02.090490 28453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:02.090587 28453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:02.090700 28453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:02.091723 28453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260808 18:22:02.092116 28453 leader_election.cc:302] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d33ef6f55da4f95b2bbf5c33ab0f2ee; no voters:
I20260808 18:22:02.092475 28453 leader_election.cc:288] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:02.092581 28467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:02.092779 28467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [term 1 LEADER]: Becoming Leader. State: Replica: 4d33ef6f55da4f95b2bbf5c33ab0f2ee, State: Running, Role: LEADER
I20260808 18:22:02.093024 28460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:02.091322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:53527 (local address 127.0.0.1:46471)
0808 18:22:02.091457 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:02.091460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:02.091669 (+ 209us) server_negotiation.cc:407] Connection header received
0808 18:22:02.091810 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:02.091812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:02.091836 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:02.091917 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:02.092568 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:02.092571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:02.092573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:02.092584 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:02.092607 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:02.092618 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:02.092716 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:02.092898 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 18:22:02.092934 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 18:22:02.093055 28467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } }
I20260808 18:22:02.093593 28453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:02.094157 28467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d33ef6f55da4f95b2bbf5c33ab0f2ee. Latest consensus state: current_term: 1 leader_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } } }
I20260808 18:22:02.094300 28467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:02.094573 28470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d33ef6f55da4f95b2bbf5c33ab0f2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46471 } } }
I20260808 18:22:02.094677 28470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:02.094765 28477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:02.097891 28477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:02.102715 28477 catalog_manager.cc:1347] Generated new cluster ID: 6f6bab3b1e6142fbb799eaba60014010
I20260808 18:22:02.102773 28477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:02.115721 28477 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:02.116046 28477 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:02.124169 28477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d33ef6f55da4f95b2bbf5c33ab0f2ee: Generated new TSK 0
I20260808 18:22:02.124701 28477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:02.128973 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28341
2026-08-08T18:22:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:02 dist-test-slave-x4qh krb5kdc[28509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:02 dist-test-slave-x4qh krb5kdc[28509](info): set up 2 sockets
Aug 08 18:22:02 dist-test-slave-x4qh krb5kdc[28509](info): commencing operation
krb5kdc: starting...
W20260808 18:22:03.268604 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:03 dist-test-slave-x4qh krb5kdc[28509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:22:03.378105 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:03.478724 28535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:03.478993 28535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:03.479058 28535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:03.482831 28535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:03.482911 28535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:03.482949 28535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:03.483003 28535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:03.483042 28535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:03.487666 28535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:03.488517 28535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:03.489367 28535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:22:03.497046 28535 server_base.cc:1034] running on GCE node
W20260808 18:22:03.497035 28550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:03.497004 28547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:03.496973 28548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:03.497810 28535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:03.498657 28535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:03.499804 28535 hybrid_clock.cc:648] HybridClock initialized: now 1786213323499788 us; error 25 us; skew 500 ppm
Aug 08 18:22:03 dist-test-slave-x4qh krb5kdc[28509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:03.502249 28535 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:22:03.503005 28535 webserver.cc:460] Webserver started at http://127.0.0.1:40291/ using document root <none> and password file <none>
I20260808 18:22:03.503499 28535 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:03.503556 28535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:03.503717 28535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:03.505477 28535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "ae4df49c893d4e27bc1f5cfc9a11707d"
format_stamp: "Formatted at 2026-08-08 18:22:03 on dist-test-slave-x4qh"
server_key: "f3258c818fd393d83feaa46fa29d50d6"
server_key_iv: "52a27f4836023aaafe29fda6b3ac2ebb"
server_key_version: "encryptionkey@0"
I20260808 18:22:03.506116 28535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "ae4df49c893d4e27bc1f5cfc9a11707d"
format_stamp: "Formatted at 2026-08-08 18:22:03 on dist-test-slave-x4qh"
server_key: "f3258c818fd393d83feaa46fa29d50d6"
server_key_iv: "52a27f4836023aaafe29fda6b3ac2ebb"
server_key_version: "encryptionkey@0"
I20260808 18:22:03.510828 28535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 18:22:03.513370 28559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:03.514389 28535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:03.514520 28535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "ae4df49c893d4e27bc1f5cfc9a11707d"
format_stamp: "Formatted at 2026-08-08 18:22:03 on dist-test-slave-x4qh"
server_key: "f3258c818fd393d83feaa46fa29d50d6"
server_key_iv: "52a27f4836023aaafe29fda6b3ac2ebb"
server_key_version: "encryptionkey@0"
I20260808 18:22:03.514618 28535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:03.527786 28535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:03.530088 28535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:03.530249 28535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:03.536276 28535 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43875
I20260808 18:22:03.536355 28647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43875 every 8 connection(s)
I20260808 18:22:03.537109 28535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:03.539513 28648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:03.543176 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28535
I20260808 18:22:03.543335 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:03.543606 27723 external_mini_cluster.cc:1428] Setting key d90fa6aba5f9b9f215c08e4588b77afc
I20260808 18:22:03.543949 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d: Bootstrap starting.
I20260808 18:22:03.546188 28648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:03.546860 28648 log.cc:824] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:03.548520 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d: No bootstrap required, opened a new log
Aug 08 18:22:03 dist-test-slave-x4qh krb5kdc[28509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 18:22:03.552083 28648 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43875 } }
I20260808 18:22:03.552266 28648 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:03.552349 28648 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4df49c893d4e27bc1f5cfc9a11707d, State: Initialized, Role: FOLLOWER
I20260808 18:22:03.552822 28648 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43875 } }
I20260808 18:22:03.552954 28648 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:03.553042 28648 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:03.553136 28648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:03.553874 28648 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43875 } }
I20260808 18:22:03.554193 28648 leader_election.cc:302] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4df49c893d4e27bc1f5cfc9a11707d; no voters:
I20260808 18:22:03.554525 28648 leader_election.cc:288] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:03.554556 28653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:03.554738 28653 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [term 1 LEADER]: Becoming Leader. State: Replica: ae4df49c893d4e27bc1f5cfc9a11707d, State: Running, Role: LEADER
I20260808 18:22:03.555051 28653 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43875 } }
I20260808 18:22:03.555677 28648 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:03.556780 28656 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4df49c893d4e27bc1f5cfc9a11707d. Latest consensus state: current_term: 1 leader_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43875 } } }
I20260808 18:22:03.556931 28656 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:03.557401 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:03.544812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:57959 (local address 127.0.0.1:43875)
0808 18:22:03.545206 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:03.545219 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:03.545250 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:22:03.545862 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:03.545877 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:03.546161 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:03.546474 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:03.547290 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:03.548194 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:03.549014 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:03.549437 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:03.549598 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:03.552173 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:03.552177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:03.552188 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:03.552212 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:03.553990 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:03.554534 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:03.554540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:03.554545 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:03.554707 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:03.555100 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:03.555102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:03.555103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:03.555616 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:03.555823 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:03.556155 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 18:22:03.556398 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":133,"threads_started":1}
I20260808 18:22:03.557328 28660 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4df49c893d4e27bc1f5cfc9a11707d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43875 } } }
I20260808 18:22:03.557880 28660 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:03.557916 28669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:03.560333 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:03.560930 28669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:03.566200 28669 catalog_manager.cc:1347] Generated new cluster ID: 8073894e16cc4baf8c64d61c788f7888
I20260808 18:22:03.566267 28669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:03.575307 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:03.563738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:35699 (local address 127.0.0.1:43875)
0808 18:22:03.563838 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:03.563842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:03.567671 (+ 3829us) server_negotiation.cc:407] Connection header received
0808 18:22:03.567783 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:03.567786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:03.567834 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:03.567905 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:03.568682 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:03.569362 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:03.570429 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:03.570758 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:03.570848 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:03.572315 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:03.572319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:03.572321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:03.572340 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:03.573714 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:03.574201 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:03.574204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:03.574206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:03.574272 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:03.574578 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:03.574580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:03.574582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:03.574733 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:03.574851 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:03.575111 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 18:22:03.575187 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:22:03.581346 28669 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:03.582345 28669 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:03.591630 28669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4df49c893d4e27bc1f5cfc9a11707d: Generated new TSK 0
I20260808 18:22:03.592258 28669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:03.599887 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28535
W20260808 18:22:03.614869 28503 connection.cc:535] client connection to 127.0.0.1:43875 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43875): Connection reset by peer (error 104)
2026-08-08T18:22:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:03 dist-test-slave-x4qh krb5kdc[28703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:03 dist-test-slave-x4qh krb5kdc[28703](info): set up 2 sockets
Aug 08 18:22:03 dist-test-slave-x4qh krb5kdc[28703](info): commencing operation
krb5kdc: starting...
W20260808 18:22:04.752135 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:04 dist-test-slave-x4qh krb5kdc[28703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:22:04.866109 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:04.968945 28723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:04.969132 28723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:04.969183 28723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:04.972712 28723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:04.972759 28723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:04.972777 28723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:04.972834 28723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:04.972859 28723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:04.977306 28723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:04.978081 28723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:04.978909 28723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:04.986663 28736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:04.986629 28731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:04.986629 28734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:04.986841 28723 server_base.cc:1034] running on GCE node
I20260808 18:22:04.987591 28723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:04.988462 28723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:04.989620 28723 hybrid_clock.cc:648] HybridClock initialized: now 1786213324989602 us; error 28 us; skew 500 ppm
Aug 08 18:22:04 dist-test-slave-x4qh krb5kdc[28703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213324, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:04.996091 28723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:22:04.996904 28723 webserver.cc:460] Webserver started at http://127.0.0.1:40321/ using document root <none> and password file <none>
I20260808 18:22:04.997380 28723 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:04.997434 28723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:04.997653 28723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:04.999481 28723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "fb52b52cd84748e6b9568b9661677b40"
format_stamp: "Formatted at 2026-08-08 18:22:04 on dist-test-slave-x4qh"
server_key: "da6cc775e5fd90ebe8459aa42eb4df6b"
server_key_iv: "989fe0478324e4ccb9529a29937f9445"
server_key_version: "encryptionkey@0"
I20260808 18:22:04.999966 28723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "fb52b52cd84748e6b9568b9661677b40"
format_stamp: "Formatted at 2026-08-08 18:22:04 on dist-test-slave-x4qh"
server_key: "da6cc775e5fd90ebe8459aa42eb4df6b"
server_key_iv: "989fe0478324e4ccb9529a29937f9445"
server_key_version: "encryptionkey@0"
I20260808 18:22:05.002920 28723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:05.005028 28749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:05.006108 28723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 18:22:05.006266 28723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "fb52b52cd84748e6b9568b9661677b40"
format_stamp: "Formatted at 2026-08-08 18:22:04 on dist-test-slave-x4qh"
server_key: "da6cc775e5fd90ebe8459aa42eb4df6b"
server_key_iv: "989fe0478324e4ccb9529a29937f9445"
server_key_version: "encryptionkey@0"
I20260808 18:22:05.006381 28723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:05.022697 28723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:05.025041 28723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:05.025254 28723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:05.031716 28723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40297
I20260808 18:22:05.031733 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40297 every 8 connection(s)
I20260808 18:22:05.032569 28723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:05.034924 28831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:05.039834 28831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40: Bootstrap starting.
I20260808 18:22:05.041874 28831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:05.042593 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28723
I20260808 18:22:05.042589 28831 log.cc:824] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:05.042743 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:05.043037 27723 external_mini_cluster.cc:1428] Setting key f046ed5fcfd7bac1c26fb08e049ef541
I20260808 18:22:05.044664 28831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40: No bootstrap required, opened a new log
I20260808 18:22:05.048234 28831 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb52b52cd84748e6b9568b9661677b40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40297 } }
I20260808 18:22:05.048480 28831 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:05.048550 28831 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb52b52cd84748e6b9568b9661677b40, State: Initialized, Role: FOLLOWER
I20260808 18:22:05.048920 28831 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb52b52cd84748e6b9568b9661677b40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40297 } }
I20260808 18:22:05.049072 28831 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:05.049160 28831 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:05.049530 28831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:05.050395 28831 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb52b52cd84748e6b9568b9661677b40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40297 } }
I20260808 18:22:05.050741 28831 leader_election.cc:302] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb52b52cd84748e6b9568b9661677b40; no voters:
I20260808 18:22:05.051038 28831 leader_election.cc:288] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:05.051118 28842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:05.051273 28842 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [term 1 LEADER]: Becoming Leader. State: Replica: fb52b52cd84748e6b9568b9661677b40, State: Running, Role: LEADER
I20260808 18:22:05.051412 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:05.044391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:58295 (local address 127.0.0.1:40297)
0808 18:22:05.044747 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:05.044764 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:05.044809 (+ 45us) server_negotiation.cc:407] Connection header received
0808 18:22:05.045365 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:05.045379 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:05.045665 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:05.045916 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:05.046269 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:05.047122 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:05.048273 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:05.048828 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:05.048938 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:05.049756 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:05.049761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:05.049771 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:05.050122 (+ 351us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:05.050166 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:05.050185 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:05.050278 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:05.050540 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 18:22:05.050644 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":127,"threads_started":1}
I20260808 18:22:05.051600 28842 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb52b52cd84748e6b9568b9661677b40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40297 } }
I20260808 18:22:05.052170 28831 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:05.052682 28842 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb52b52cd84748e6b9568b9661677b40. Latest consensus state: current_term: 1 leader_uuid: "fb52b52cd84748e6b9568b9661677b40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb52b52cd84748e6b9568b9661677b40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40297 } } }
I20260808 18:22:05.052863 28842 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:05.053058 28844 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb52b52cd84748e6b9568b9661677b40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb52b52cd84748e6b9568b9661677b40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40297 } } }
I20260808 18:22:05.053188 28844 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:05.053329 28856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:05.054361 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:05.058009 28856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:05.061034 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:05.056373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:57479 (local address 127.0.0.1:40297)
0808 18:22:05.056488 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:05.056493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:05.056872 (+ 379us) server_negotiation.cc:407] Connection header received
0808 18:22:05.057011 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:05.057014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:05.057079 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:05.057144 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:05.057462 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:05.058136 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:05.059211 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:05.059510 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:05.059646 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:05.060395 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:05.060399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:05.060401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:05.060412 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:05.060435 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:05.060445 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:05.060847 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:05.060896 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 18:22:05.060924 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 18:22:05.063570 28856 catalog_manager.cc:1347] Generated new cluster ID: 903b4b872a9f4b16b65e4c2c5246d1e6
I20260808 18:22:05.063645 28856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:05.070652 28856 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:05.071362 28856 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:05.074587 28856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb52b52cd84748e6b9568b9661677b40: Generated new TSK 0
I20260808 18:22:05.075088 28856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:05.082139 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28723
W20260808 18:22:05.103554 28695 connection.cc:535] client connection to 127.0.0.1:40297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40297): Connection reset by peer (error 104)
2026-08-08T18:22:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:05 dist-test-slave-x4qh krb5kdc[28887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:05 dist-test-slave-x4qh krb5kdc[28887](info): set up 2 sockets
Aug 08 18:22:05 dist-test-slave-x4qh krb5kdc[28887](info): commencing operation
krb5kdc: starting...
W20260808 18:22:06.235388 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:06 dist-test-slave-x4qh krb5kdc[28887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 18:22:06.347738 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:06.447154 28908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:06.447374 28908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:06.447494 28908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:06.450950 28908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:06.451021 28908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:06.451054 28908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:06.451092 28908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:06.451120 28908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:06.455461 28908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39089
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:06.456357 28908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:06.457244 28908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:06.464825 28930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:06.464846 28927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:06.464829 28933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:06.464972 28908 server_base.cc:1034] running on GCE node
I20260808 18:22:06.465639 28908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:06.466523 28908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:06.467674 28908 hybrid_clock.cc:648] HybridClock initialized: now 1786213326467653 us; error 31 us; skew 500 ppm
Aug 08 18:22:06 dist-test-slave-x4qh krb5kdc[28887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213326, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:06.470009 28908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 18:22:06.470723 28908 webserver.cc:460] Webserver started at http://127.0.0.1:38211/ using document root <none> and password file <none>
I20260808 18:22:06.471174 28908 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:06.471253 28908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:06.471499 28908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:06.473198 28908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "3d8681d7f3d745bfa70aa31ff7a069cf"
format_stamp: "Formatted at 2026-08-08 18:22:06 on dist-test-slave-x4qh"
server_key: "f9ce6fc61de09f87a3ffc9fb8a3c6cd4"
server_key_iv: "55962f4637743c56aff3c48d42e1f908"
server_key_version: "encryptionkey@0"
I20260808 18:22:06.473766 28908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "3d8681d7f3d745bfa70aa31ff7a069cf"
format_stamp: "Formatted at 2026-08-08 18:22:06 on dist-test-slave-x4qh"
server_key: "f9ce6fc61de09f87a3ffc9fb8a3c6cd4"
server_key_iv: "55962f4637743c56aff3c48d42e1f908"
server_key_version: "encryptionkey@0"
I20260808 18:22:06.477126 28908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:06.479084 28948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:06.480110 28908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:06.480244 28908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "3d8681d7f3d745bfa70aa31ff7a069cf"
format_stamp: "Formatted at 2026-08-08 18:22:06 on dist-test-slave-x4qh"
server_key: "f9ce6fc61de09f87a3ffc9fb8a3c6cd4"
server_key_iv: "55962f4637743c56aff3c48d42e1f908"
server_key_version: "encryptionkey@0"
I20260808 18:22:06.480336 28908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:06.484251 28908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:06.486598 28908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:06.486774 28908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:06.493103 28908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39089
I20260808 18:22:06.493130 29032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39089 every 8 connection(s)
I20260808 18:22:06.493883 28908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:06.496297 29033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:06.501044 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf: Bootstrap starting.
I20260808 18:22:06.503057 29033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:06.503192 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 28908
I20260808 18:22:06.503346 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:06.503612 27723 external_mini_cluster.cc:1428] Setting key d3e445ec37cab5ad89d5e3d1a01646fe
I20260808 18:22:06.503863 29033 log.cc:824] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:06.505834 29033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf: No bootstrap required, opened a new log
I20260808 18:22:06.508097 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:06.504610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:38985 (local address 127.0.0.1:39089)
0808 18:22:06.504985 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:06.504998 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:06.505028 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:22:06.505609 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:06.505624 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:06.505982 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:06.506223 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:06.506996 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:06.507000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:06.507011 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:06.507021 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:06.507066 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:06.507083 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:06.507164 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:06.507309 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 18:22:06.507487 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":114,"threads_started":1}
I20260808 18:22:06.509756 29033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39089 } }
I20260808 18:22:06.510002 29033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:06.510058 29033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8681d7f3d745bfa70aa31ff7a069cf, State: Initialized, Role: FOLLOWER
I20260808 18:22:06.510442 29033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39089 } }
I20260808 18:22:06.510584 29033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:06.510651 29033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:06.510674 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:06.510740 29033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:06.511801 29033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39089 } }
I20260808 18:22:06.512161 29033 leader_election.cc:302] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d8681d7f3d745bfa70aa31ff7a069cf; no voters:
I20260808 18:22:06.512528 29033 leader_election.cc:288] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:06.512624 29053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:06.512825 29053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [term 1 LEADER]: Becoming Leader. State: Replica: 3d8681d7f3d745bfa70aa31ff7a069cf, State: Running, Role: LEADER
I20260808 18:22:06.513155 29053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39089 } }
I20260808 18:22:06.513660 29033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:06.514111 29053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39089 } } }
I20260808 18:22:06.514266 29053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:06.514510 29053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d8681d7f3d745bfa70aa31ff7a069cf. Latest consensus state: current_term: 1 leader_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8681d7f3d745bfa70aa31ff7a069cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39089 } } }
I20260808 18:22:06.514618 29053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:06.515878 29073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:06.516544 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:06.512929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.93:42287 (local address 127.0.0.1:39089)
0808 18:22:06.513027 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:06.513031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:06.515058 (+ 2027us) server_negotiation.cc:407] Connection header received
0808 18:22:06.515147 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:06.515152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:06.515189 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:06.515277 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:06.516146 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:06.516152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:06.516154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:06.516168 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:06.516196 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:06.516212 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:06.516301 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:06.516396 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 18:22:06.516425 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 18:22:06.518993 29073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:06.523602 29073 catalog_manager.cc:1347] Generated new cluster ID: c2f51d91d5ea40399a4447b2cd17a2b9
I20260808 18:22:06.523659 29073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:06.529603 29073 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:06.529860 29073 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:06.534489 29073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d8681d7f3d745bfa70aa31ff7a069cf: Generated new TSK 0
I20260808 18:22:06.535068 29073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:06.544834 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 28908
2026-08-08T18:22:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:06 dist-test-slave-x4qh krb5kdc[29097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:06 dist-test-slave-x4qh krb5kdc[29097](info): set up 2 sockets
Aug 08 18:22:06 dist-test-slave-x4qh krb5kdc[29097](info): commencing operation
krb5kdc: starting...
W20260808 18:22:07.708699 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:07 dist-test-slave-x4qh krb5kdc[29097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:07.822115 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35011
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:35011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:07.939082 29119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:07.939355 29119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:07.939450 29119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:07.943084 29119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:07.943150 29119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:07.943183 29119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:07.943252 29119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:07.943288 29119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:07.948454 29119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:35011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:07.949363 29119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:07.950264 29119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:07.958052 29130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:07.958041 29129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:07.958056 29133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:07.958277 29119 server_base.cc:1034] running on GCE node
I20260808 18:22:07.958890 29119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:07.959833 29119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:07.961004 29119 hybrid_clock.cc:648] HybridClock initialized: now 1786213327960982 us; error 33 us; skew 500 ppm
Aug 08 18:22:07 dist-test-slave-x4qh krb5kdc[29097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213327, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:07.963711 29119 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:07.964547 29119 webserver.cc:460] Webserver started at http://127.27.18.254:35203/ using document root <none> and password file <none>
I20260808 18:22:07.965070 29119 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:07.965157 29119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:07.965375 29119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:07.967163 29119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "0876cdb28bce4996a9e4536b68c32388"
format_stamp: "Formatted at 2026-08-08 18:22:07 on dist-test-slave-x4qh"
server_key: "b72b20bebc3a1d9f03268d457011e7ae"
server_key_iv: "f4c81d3604b5534c82cef546b3a1fe5b"
server_key_version: "encryptionkey@0"
I20260808 18:22:07.967815 29119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "0876cdb28bce4996a9e4536b68c32388"
format_stamp: "Formatted at 2026-08-08 18:22:07 on dist-test-slave-x4qh"
server_key: "b72b20bebc3a1d9f03268d457011e7ae"
server_key_iv: "f4c81d3604b5534c82cef546b3a1fe5b"
server_key_version: "encryptionkey@0"
I20260808 18:22:07.971089 29119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:07.973568 29144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:07.974599 29119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:07.974725 29119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "0876cdb28bce4996a9e4536b68c32388"
format_stamp: "Formatted at 2026-08-08 18:22:07 on dist-test-slave-x4qh"
server_key: "b72b20bebc3a1d9f03268d457011e7ae"
server_key_iv: "f4c81d3604b5534c82cef546b3a1fe5b"
server_key_version: "encryptionkey@0"
I20260808 18:22:07.974830 29119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:07.987552 29119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:07.990461 29119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:07.990658 29119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:07.997151 29119 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:35011
I20260808 18:22:07.997164 29229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:35011 every 8 connection(s)
I20260808 18:22:07.997982 29119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:08.000370 29232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:08.005014 29232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388: Bootstrap starting.
I20260808 18:22:08.007154 29232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:08.007789 29232 log.cc:824] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:08.007993 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29119
I20260808 18:22:08.008152 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:08.008476 27723 external_mini_cluster.cc:1428] Setting key 9d010a94961037b5290ca76f5a3bcd84
I20260808 18:22:08.009857 29232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388: No bootstrap required, opened a new log
I20260808 18:22:08.014204 29232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876cdb28bce4996a9e4536b68c32388" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35011 } }
I20260808 18:22:08.014436 29232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:08.014514 29232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0876cdb28bce4996a9e4536b68c32388, State: Initialized, Role: FOLLOWER
I20260808 18:22:08.014887 29232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876cdb28bce4996a9e4536b68c32388" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35011 } }
I20260808 18:22:08.015035 29232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:08.015116 29232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:08.015235 29232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:08.016119 29232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876cdb28bce4996a9e4536b68c32388" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35011 } }
I20260808 18:22:08.016448 29232 leader_election.cc:302] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0876cdb28bce4996a9e4536b68c32388; no voters:
I20260808 18:22:08.016762 29232 leader_election.cc:288] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:08.016841 29243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:08.017010 29243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [term 1 LEADER]: Becoming Leader. State: Replica: 0876cdb28bce4996a9e4536b68c32388, State: Running, Role: LEADER
I20260808 18:22:08.017378 29243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876cdb28bce4996a9e4536b68c32388" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35011 } }
Aug 08 18:22:08 dist-test-slave-x4qh krb5kdc[29097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:08.017958 29232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:08.019171 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0876cdb28bce4996a9e4536b68c32388. Latest consensus state: current_term: 1 leader_uuid: "0876cdb28bce4996a9e4536b68c32388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876cdb28bce4996a9e4536b68c32388" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35011 } } }
I20260808 18:22:08.019321 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:08.019359 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0876cdb28bce4996a9e4536b68c32388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876cdb28bce4996a9e4536b68c32388" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35011 } } }
I20260808 18:22:08.019452 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:08.019773 29255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:08.022449 29255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:08.024067 29241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:08.009504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47195 (local address 127.27.18.254:35011)
0808 18:22:08.009922 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:08.009940 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:08.009978 (+ 38us) server_negotiation.cc:407] Connection header received
0808 18:22:08.010680 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:08.010701 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:08.011098 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:08.011466 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:08.012125 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:08.013690 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:08.014558 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:08.015163 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:08.015351 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:08.018273 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:08.018278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:08.018292 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:08.018322 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:08.020505 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:08.021079 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:08.021084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:08.021091 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:08.021223 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:08.021595 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:08.021598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:08.021599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:08.022032 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:08.022219 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:08.023024 (+ 805us) server_negotiation.cc:299] Negotiation successful
0808 18:22:08.023223 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":104,"threads_started":1}
I20260808 18:22:08.027040 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:08.027446 29255 catalog_manager.cc:1347] Generated new cluster ID: 1799263ddfa4418f97b38485781e0905
I20260808 18:22:08.027490 29255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:08.039912 29241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:08.029308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36813 (local address 127.27.18.254:35011)
0808 18:22:08.029391 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:08.029395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:08.031649 (+ 2254us) server_negotiation.cc:407] Connection header received
0808 18:22:08.031840 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:08.031843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:08.031882 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:08.031949 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:08.032760 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:08.033552 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:08.034647 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:08.034979 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:08.035064 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:08.036557 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:08.036561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:08.036563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:08.036586 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:08.038161 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:08.038677 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:08.038680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:08.038682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:08.038758 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:08.039096 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:08.039099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:08.039100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:08.039274 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:08.039422 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:08.039708 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 18:22:08.039784 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 18:22:08.043192 29255 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:08.043964 29255 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:08.050499 29255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0876cdb28bce4996a9e4536b68c32388: Generated new TSK 0
I20260808 18:22:08.050983 29255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:08.054086 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29119
W20260808 18:22:08.070158 29094 connection.cc:535] client connection to 127.27.18.254:35011 recv error: Network error: failed to read from TLS socket (remote: 127.27.18.254:35011): Connection reset by peer (error 104)
2026-08-08T18:22:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:08 dist-test-slave-x4qh krb5kdc[29290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:08 dist-test-slave-x4qh krb5kdc[29290](info): set up 2 sockets
Aug 08 18:22:08 dist-test-slave-x4qh krb5kdc[29290](info): commencing operation
krb5kdc: starting...
W20260808 18:22:09.209486 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:09 dist-test-slave-x4qh krb5kdc[29290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:09.322050 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33031
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:33031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:09.426416 29309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:09.426682 29309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:09.426776 29309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:09.430531 29309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:09.430612 29309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:09.430648 29309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:09.430721 29309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:09.430757 29309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:09.436124 29309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:33031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:33031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:09.437137 29309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:09.438102 29309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:09.445978 29330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:09.446005 29324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:09.446043 29322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:09.446357 29309 server_base.cc:1034] running on GCE node
I20260808 18:22:09.446888 29309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:09.447939 29309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:09.449124 29309 hybrid_clock.cc:648] HybridClock initialized: now 1786213329449111 us; error 33 us; skew 500 ppm
Aug 08 18:22:09 dist-test-slave-x4qh krb5kdc[29290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213329, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:09.451617 29309 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:09.452441 29309 webserver.cc:460] Webserver started at http://127.27.18.254:41139/ using document root <none> and password file <none>
I20260808 18:22:09.452934 29309 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:09.453018 29309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:09.453230 29309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:09.455073 29309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "43d6802b44834d6786beabd19fd9889a"
format_stamp: "Formatted at 2026-08-08 18:22:09 on dist-test-slave-x4qh"
server_key: "28f193e30ff97934553122f483001197"
server_key_iv: "f8a567059c3556de180bf292520d1a49"
server_key_version: "encryptionkey@0"
I20260808 18:22:09.455614 29309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "43d6802b44834d6786beabd19fd9889a"
format_stamp: "Formatted at 2026-08-08 18:22:09 on dist-test-slave-x4qh"
server_key: "28f193e30ff97934553122f483001197"
server_key_iv: "f8a567059c3556de180bf292520d1a49"
server_key_version: "encryptionkey@0"
I20260808 18:22:09.459003 29309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 18:22:09.461236 29340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:09.462282 29309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:09.462419 29309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "43d6802b44834d6786beabd19fd9889a"
format_stamp: "Formatted at 2026-08-08 18:22:09 on dist-test-slave-x4qh"
server_key: "28f193e30ff97934553122f483001197"
server_key_iv: "f8a567059c3556de180bf292520d1a49"
server_key_version: "encryptionkey@0"
I20260808 18:22:09.462525 29309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:09.483678 29309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:09.486279 29309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:09.486477 29309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:09.493502 29309 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:33031
I20260808 18:22:09.493527 29430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:33031 every 8 connection(s)
I20260808 18:22:09.494524 29309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:09.497032 29431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:09.498088 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29309
I20260808 18:22:09.498243 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:09.498556 27723 external_mini_cluster.cc:1428] Setting key 02dbb9c925d3531e7f1b08dea92a3bbd
I20260808 18:22:09.502763 29431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a: Bootstrap starting.
I20260808 18:22:09.504904 29431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:09.505664 29431 log.cc:824] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:09.507958 29431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a: No bootstrap required, opened a new log
I20260808 18:22:09.508540 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:09.499908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39771 (local address 127.27.18.254:33031)
0808 18:22:09.500336 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:09.500350 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:09.500381 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:22:09.500951 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:09.500967 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:09.501257 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:09.501565 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:09.501930 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:09.502859 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:09.503790 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:09.504204 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:09.504401 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:09.505312 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:09.505317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:09.505328 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:09.505765 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:09.505835 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:09.505866 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:09.506021 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:09.507409 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0808 18:22:09.507549 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":131,"threads_started":1}
I20260808 18:22:09.513044 29431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43d6802b44834d6786beabd19fd9889a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33031 } }
I20260808 18:22:09.513316 29431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:09.513386 29431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d6802b44834d6786beabd19fd9889a, State: Initialized, Role: FOLLOWER
I20260808 18:22:09.513806 29431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43d6802b44834d6786beabd19fd9889a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33031 } }
I20260808 18:22:09.513968 29431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:09.513998 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:09.514036 29431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:09.514127 29431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:09.515230 29431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43d6802b44834d6786beabd19fd9889a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33031 } }
I20260808 18:22:09.515627 29431 leader_election.cc:302] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43d6802b44834d6786beabd19fd9889a; no voters:
I20260808 18:22:09.517869 29431 leader_election.cc:288] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:09.517961 29443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:09.518333 29443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [term 1 LEADER]: Becoming Leader. State: Replica: 43d6802b44834d6786beabd19fd9889a, State: Running, Role: LEADER
I20260808 18:22:09.518690 29443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43d6802b44834d6786beabd19fd9889a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33031 } }
I20260808 18:22:09.519279 29431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:09.519948 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43d6802b44834d6786beabd19fd9889a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43d6802b44834d6786beabd19fd9889a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33031 } } }
I20260808 18:22:09.520052 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:09.520254 29443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43d6802b44834d6786beabd19fd9889a. Latest consensus state: current_term: 1 leader_uuid: "43d6802b44834d6786beabd19fd9889a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43d6802b44834d6786beabd19fd9889a" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 33031 } } }
I20260808 18:22:09.520355 29443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:09.523144 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:09.518117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37735 (local address 127.27.18.254:33031)
0808 18:22:09.518223 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:09.518229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:09.518538 (+ 309us) server_negotiation.cc:407] Connection header received
0808 18:22:09.518738 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:09.518741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:09.518815 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:09.518865 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:09.519260 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:09.520010 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:09.521196 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:09.521490 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:09.521595 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:09.522610 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:09.522615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:09.522618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:09.522631 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:09.522659 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:09.522672 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:09.522759 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:09.522989 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 18:22:09.523023 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 18:22:09.525260 29466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:09.525341 29466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:09.525401 29456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:09.526260 29456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:09.534555 29456 catalog_manager.cc:1347] Generated new cluster ID: 5b7e2f9d324c4a558672f127014dace0
I20260808 18:22:09.534617 29456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:09.547320 29456 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:09.548127 29456 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:09.561592 29456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43d6802b44834d6786beabd19fd9889a: Generated new TSK 0
I20260808 18:22:09.562283 29456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:09.567507 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29309
W20260808 18:22:09.580402 29288 connection.cc:535] client connection to 127.27.18.254:33031 recv error: Network error: failed to read from TLS socket (remote: 127.27.18.254:33031): Connection reset by peer (error 104)
2026-08-08T18:22:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:09 dist-test-slave-x4qh krb5kdc[29478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:09 dist-test-slave-x4qh krb5kdc[29478](info): set up 2 sockets
Aug 08 18:22:09 dist-test-slave-x4qh krb5kdc[29478](info): commencing operation
krb5kdc: starting...
W20260808 18:22:10.723582 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:10 dist-test-slave-x4qh krb5kdc[29478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:10.840165 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44845
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:44845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:10.944204 29498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:10.944448 29498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:10.944506 29498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:10.948207 29498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:10.948256 29498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:10.948276 29498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:10.948328 29498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:10.948360 29498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:10.952898 29498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:44845
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:44845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:10.953745 29498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:10.954631 29498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:10.962196 29512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:10.962203 29507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:10.962415 29498 server_base.cc:1034] running on GCE node
W20260808 18:22:10.962226 29509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:10.963001 29498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:10.964002 29498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:10.965171 29498 hybrid_clock.cc:648] HybridClock initialized: now 1786213330965150 us; error 49 us; skew 500 ppm
Aug 08 18:22:10 dist-test-slave-x4qh krb5kdc[29478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213330, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:10.970144 29498 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:10.970855 29498 webserver.cc:460] Webserver started at http://127.27.18.254:34117/ using document root <none> and password file <none>
I20260808 18:22:10.971320 29498 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:10.971414 29498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:10.971621 29498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:10.973315 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "18407cd13be342d4a8d33aa5c0a9c391"
format_stamp: "Formatted at 2026-08-08 18:22:10 on dist-test-slave-x4qh"
server_key: "b7d91fc298d2e421177800ab21d2c08f"
server_key_iv: "d5ec90d6514e79da22b88c94266acc56"
server_key_version: "encryptionkey@0"
I20260808 18:22:10.973836 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "18407cd13be342d4a8d33aa5c0a9c391"
format_stamp: "Formatted at 2026-08-08 18:22:10 on dist-test-slave-x4qh"
server_key: "b7d91fc298d2e421177800ab21d2c08f"
server_key_iv: "d5ec90d6514e79da22b88c94266acc56"
server_key_version: "encryptionkey@0"
I20260808 18:22:10.977272 29498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 18:22:10.979349 29524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:10.980479 29498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:10.980607 29498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "18407cd13be342d4a8d33aa5c0a9c391"
format_stamp: "Formatted at 2026-08-08 18:22:10 on dist-test-slave-x4qh"
server_key: "b7d91fc298d2e421177800ab21d2c08f"
server_key_iv: "d5ec90d6514e79da22b88c94266acc56"
server_key_version: "encryptionkey@0"
I20260808 18:22:10.980692 29498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:10.984766 29498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:10.987713 29498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:10.987896 29498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:10.994122 29498 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:44845
I20260808 18:22:10.994130 29606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:44845 every 8 connection(s)
I20260808 18:22:10.994917 29498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:10.995409 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29498
I20260808 18:22:10.995551 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:10.995748 27723 external_mini_cluster.cc:1428] Setting key 9df335e8b2f8ce0b3d522a810bf8eaa5
I20260808 18:22:10.997848 29607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:11.000488 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:10.996893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33463 (local address 127.27.18.254:44845)
0808 18:22:10.997340 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:10.997355 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:10.997425 (+ 70us) server_negotiation.cc:407] Connection header received
0808 18:22:10.998064 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:10.998081 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:10.998326 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:10.998695 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:10.999449 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:10.999454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:10.999464 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:10.999475 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:10.999520 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:10.999537 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:10.999640 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:10.999756 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 18:22:10.999899 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260808 18:22:11.002918 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:11.003504 29607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391: Bootstrap starting.
I20260808 18:22:11.006039 29607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:11.006553 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:11.005110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36675 (local address 127.27.18.254:44845)
0808 18:22:11.005215 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:11.005220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:11.005375 (+ 155us) server_negotiation.cc:407] Connection header received
0808 18:22:11.005489 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:11.005491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:11.005528 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:11.005578 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:11.006161 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:11.006165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:11.006167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:11.006180 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:11.006205 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:11.006214 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:11.006274 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:11.006403 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 18:22:11.006430 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:22:11.006867 29607 log.cc:824] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:11.008550 29607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391: No bootstrap required, opened a new log
I20260808 18:22:11.011655 29607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18407cd13be342d4a8d33aa5c0a9c391" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44845 } }
I20260808 18:22:11.011861 29607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:11.011945 29607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18407cd13be342d4a8d33aa5c0a9c391, State: Initialized, Role: FOLLOWER
I20260808 18:22:11.012385 29607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18407cd13be342d4a8d33aa5c0a9c391" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44845 } }
I20260808 18:22:11.012527 29607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:11.012586 29607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:11.012681 29607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:11.013437 29607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18407cd13be342d4a8d33aa5c0a9c391" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44845 } }
I20260808 18:22:11.013767 29607 leader_election.cc:302] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18407cd13be342d4a8d33aa5c0a9c391; no voters:
I20260808 18:22:11.014055 29607 leader_election.cc:288] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:11.014190 29633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:11.014420 29633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [term 1 LEADER]: Becoming Leader. State: Replica: 18407cd13be342d4a8d33aa5c0a9c391, State: Running, Role: LEADER
I20260808 18:22:11.014729 29633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18407cd13be342d4a8d33aa5c0a9c391" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44845 } }
I20260808 18:22:11.015168 29607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:11.016340 29634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18407cd13be342d4a8d33aa5c0a9c391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18407cd13be342d4a8d33aa5c0a9c391" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44845 } } }
I20260808 18:22:11.016477 29634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:11.016891 29648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:11.016875 29637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18407cd13be342d4a8d33aa5c0a9c391. Latest consensus state: current_term: 1 leader_uuid: "18407cd13be342d4a8d33aa5c0a9c391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18407cd13be342d4a8d33aa5c0a9c391" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 44845 } } }
I20260808 18:22:11.019472 29637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:11.019830 29648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:11.024937 29648 catalog_manager.cc:1347] Generated new cluster ID: 679cae1fa31a4921b20b22e54e5dea3b
I20260808 18:22:11.024996 29648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:11.045728 29648 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:11.046005 29648 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:11.060141 29648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18407cd13be342d4a8d33aa5c0a9c391: Generated new TSK 0
I20260808 18:22:11.060819 29648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:11.063580 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29498
2026-08-08T18:22:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:11 dist-test-slave-x4qh krb5kdc[29662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:11 dist-test-slave-x4qh krb5kdc[29662](info): set up 2 sockets
Aug 08 18:22:11 dist-test-slave-x4qh krb5kdc[29662](info): commencing operation
krb5kdc: starting...
W20260808 18:22:12.213147 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:12 dist-test-slave-x4qh krb5kdc[29662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:12.324496 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41923
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:40345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:41923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:12.426379 29682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:12.426589 29682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:12.426627 29682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:12.430126 29682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:12.430172 29682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:12.430190 29682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:12.430233 29682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:12.430274 29682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:12.434808 29682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:40345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:41923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:41923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:12.435685 29682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:12.436575 29682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:12.444489 29701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:12.444526 29697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:12.444636 29682 server_base.cc:1034] running on GCE node
W20260808 18:22:12.444531 29696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:12.445206 29682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:12.446149 29682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:12.447340 29682 hybrid_clock.cc:648] HybridClock initialized: now 1786213332447322 us; error 29 us; skew 500 ppm
Aug 08 18:22:12 dist-test-slave-x4qh krb5kdc[29662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213332, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:12.449823 29682 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:12.450552 29682 webserver.cc:460] Webserver started at http://127.27.18.254:39731/ using document root <none> and password file <none>
I20260808 18:22:12.451035 29682 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:12.451115 29682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:12.451313 29682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:12.453069 29682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "48a6cd7912434ff89467545ab94bb767"
format_stamp: "Formatted at 2026-08-08 18:22:12 on dist-test-slave-x4qh"
server_key: "deeaaf20ecf3db797cbf2b27a60f839b"
server_key_iv: "8b53563cc12e3d0a1d235973d2f3e9b5"
server_key_version: "encryptionkey@0"
I20260808 18:22:12.453572 29682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "48a6cd7912434ff89467545ab94bb767"
format_stamp: "Formatted at 2026-08-08 18:22:12 on dist-test-slave-x4qh"
server_key: "deeaaf20ecf3db797cbf2b27a60f839b"
server_key_iv: "8b53563cc12e3d0a1d235973d2f3e9b5"
server_key_version: "encryptionkey@0"
I20260808 18:22:12.456939 29682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:12.459201 29712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:12.460346 29682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:12.460448 29682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "48a6cd7912434ff89467545ab94bb767"
format_stamp: "Formatted at 2026-08-08 18:22:12 on dist-test-slave-x4qh"
server_key: "deeaaf20ecf3db797cbf2b27a60f839b"
server_key_iv: "8b53563cc12e3d0a1d235973d2f3e9b5"
server_key_version: "encryptionkey@0"
I20260808 18:22:12.460520 29682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:12.503346 29682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:12.506426 29682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:12.506569 29682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:12.512997 29682 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:41923
I20260808 18:22:12.513025 29808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:41923 every 8 connection(s)
I20260808 18:22:12.513849 29682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:12.516319 29809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:12.520329 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29682
I20260808 18:22:12.520463 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:12.520718 27723 external_mini_cluster.cc:1428] Setting key f4c0850ac6d9f1535695010d8c25a9b1
I20260808 18:22:12.521893 29809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767: Bootstrap starting.
I20260808 18:22:12.523950 29809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:12.524693 29809 log.cc:824] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:12.526417 29809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767: No bootstrap required, opened a new log
Aug 08 18:22:12 dist-test-slave-x4qh krb5kdc[29662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:12.529641 29809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a6cd7912434ff89467545ab94bb767" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41923 } }
I20260808 18:22:12.529866 29809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:12.529939 29809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a6cd7912434ff89467545ab94bb767, State: Initialized, Role: FOLLOWER
I20260808 18:22:12.530372 29809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a6cd7912434ff89467545ab94bb767" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41923 } }
I20260808 18:22:12.530546 29809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:12.530637 29809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:12.530735 29809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:12.531562 29809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a6cd7912434ff89467545ab94bb767" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41923 } }
I20260808 18:22:12.531889 29809 leader_election.cc:302] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48a6cd7912434ff89467545ab94bb767; no voters:
I20260808 18:22:12.532198 29809 leader_election.cc:288] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:12.532316 29819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:12.532517 29819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [term 1 LEADER]: Becoming Leader. State: Replica: 48a6cd7912434ff89467545ab94bb767, State: Running, Role: LEADER
I20260808 18:22:12.532925 29819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a6cd7912434ff89467545ab94bb767" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41923 } }
I20260808 18:22:12.533474 29809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:12.534101 29819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48a6cd7912434ff89467545ab94bb767. Latest consensus state: current_term: 1 leader_uuid: "48a6cd7912434ff89467545ab94bb767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a6cd7912434ff89467545ab94bb767" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41923 } } }
I20260808 18:22:12.534204 29819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:12.534533 29830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:12.534518 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:12.521994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42611 (local address 127.27.18.254:41923)
0808 18:22:12.522396 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:12.522409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:12.522439 (+ 30us) server_negotiation.cc:407] Connection header received
0808 18:22:12.523014 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:12.523029 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:12.523318 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:12.523667 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:12.524453 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:12.525458 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:12.526265 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:12.526778 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:12.526939 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:12.529213 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:12.529218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:12.529229 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:12.529259 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:12.531037 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:12.531502 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:12.531507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:12.531513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:12.531622 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:12.531918 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:12.531921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:12.531923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:12.532497 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:12.533291 (+ 794us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:12.533420 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 18:22:12.533668 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260808 18:22:12.535434 29821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48a6cd7912434ff89467545ab94bb767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a6cd7912434ff89467545ab94bb767" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 41923 } } }
I20260808 18:22:12.535593 29821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:12.537829 29830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:12.537899 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:12.543040 29830 catalog_manager.cc:1347] Generated new cluster ID: 9d477c16a2de4dffabe1ee9615b24b83
I20260808 18:22:12.543108 29830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:12.548275 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:12.539962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39115 (local address 127.27.18.254:41923)
0808 18:22:12.540055 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:12.540059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:12.540313 (+ 254us) server_negotiation.cc:407] Connection header received
0808 18:22:12.540410 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:12.540412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:12.540450 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:12.540510 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:12.541295 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:12.542002 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:12.542951 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:12.543299 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:12.543413 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:12.544877 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:12.544881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:12.544884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:12.544909 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:12.546477 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:12.546918 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:12.546921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:12.546923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:12.547011 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:12.547278 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:12.547283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:12.547285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:12.547513 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:12.547638 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:12.547870 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 18:22:12.548115 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:22:12.555225 29830 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:12.555959 29830 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:12.561147 29830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48a6cd7912434ff89467545ab94bb767: Generated new TSK 0
I20260808 18:22:12.561702 29830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:12.569404 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29682
2026-08-08T18:22:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:12 dist-test-slave-x4qh krb5kdc[29865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:12 dist-test-slave-x4qh krb5kdc[29865](info): set up 2 sockets
Aug 08 18:22:12 dist-test-slave-x4qh krb5kdc[29865](info): commencing operation
krb5kdc: starting...
W20260808 18:22:13.732230 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:13 dist-test-slave-x4qh krb5kdc[29865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:13.841905 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35521
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:46685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:13.945678 29888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:13.945924 29888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:13.945988 29888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:13.949693 29888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:13.949767 29888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:13.949800 29888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:13.949880 29888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:13.949914 29888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:13.954332 29888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:35521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:13.955224 29888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:13.956156 29888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:13.963959 29898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:13.964102 29888 server_base.cc:1034] running on GCE node
W20260808 18:22:13.963936 29895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:13.963946 29896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:13.964772 29888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:13.965727 29888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:13.966882 29888 hybrid_clock.cc:648] HybridClock initialized: now 1786213333966859 us; error 34 us; skew 500 ppm
Aug 08 18:22:13 dist-test-slave-x4qh krb5kdc[29865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213333, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:13.969436 29888 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:13.970188 29888 webserver.cc:460] Webserver started at http://127.27.18.254:45959/ using document root <none> and password file <none>
I20260808 18:22:13.970670 29888 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:13.970754 29888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:13.970960 29888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:13.972787 29888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "d0578719333b4412a38b8ef74a45f7d0"
format_stamp: "Formatted at 2026-08-08 18:22:13 on dist-test-slave-x4qh"
server_key: "0dc3fd4dd9d0027192d41cfab9a2cdc4"
server_key_iv: "de18fab800a1b5aa6044c426dec1c7d4"
server_key_version: "encryptionkey@0"
I20260808 18:22:13.973328 29888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "d0578719333b4412a38b8ef74a45f7d0"
format_stamp: "Formatted at 2026-08-08 18:22:13 on dist-test-slave-x4qh"
server_key: "0dc3fd4dd9d0027192d41cfab9a2cdc4"
server_key_iv: "de18fab800a1b5aa6044c426dec1c7d4"
server_key_version: "encryptionkey@0"
I20260808 18:22:13.976540 29888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 18:22:13.978706 29908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:13.979801 29888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 18:22:13.979930 29888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "d0578719333b4412a38b8ef74a45f7d0"
format_stamp: "Formatted at 2026-08-08 18:22:13 on dist-test-slave-x4qh"
server_key: "0dc3fd4dd9d0027192d41cfab9a2cdc4"
server_key_iv: "de18fab800a1b5aa6044c426dec1c7d4"
server_key_version: "encryptionkey@0"
I20260808 18:22:13.980028 29888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:13.988919 29888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:13.991485 29888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:13.991665 29888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:13.998322 29888 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:35521
I20260808 18:22:13.998328 29991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:35521 every 8 connection(s)
I20260808 18:22:13.999137 29888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:14.001562 29992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:14.006765 29992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0: Bootstrap starting.
I20260808 18:22:14.007475 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 29888
I20260808 18:22:14.007634 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:14.007926 27723 external_mini_cluster.cc:1428] Setting key 27e9d767f3fa285bb8fe36d09388e7ee
I20260808 18:22:14.009366 29992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:14.010082 29992 log.cc:824] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:14.012004 29992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0: No bootstrap required, opened a new log
I20260808 18:22:14.015517 29992 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0578719333b4412a38b8ef74a45f7d0" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35521 } }
I20260808 18:22:14.015759 29992 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:14.015843 29992 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0578719333b4412a38b8ef74a45f7d0, State: Initialized, Role: FOLLOWER
I20260808 18:22:14.016175 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:14.008894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45695 (local address 127.27.18.254:35521)
0808 18:22:14.009273 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:14.009287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:14.009325 (+ 38us) server_negotiation.cc:407] Connection header received
0808 18:22:14.009979 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:14.009995 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:14.010327 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:14.010654 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:14.010974 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:14.012010 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:14.012931 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:14.013398 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:14.013565 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:14.014381 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:14.014386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:14.014397 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:14.014732 (+ 335us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:14.014780 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:14.014801 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:14.014907 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:14.015059 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 18:22:14.015248 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":114,"threads_started":1}
I20260808 18:22:14.016258 29992 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0578719333b4412a38b8ef74a45f7d0" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35521 } }
I20260808 18:22:14.016435 29992 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:14.016517 29992 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:14.016610 29992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:14.017522 29992 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0578719333b4412a38b8ef74a45f7d0" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35521 } }
I20260808 18:22:14.017859 29992 leader_election.cc:302] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0578719333b4412a38b8ef74a45f7d0; no voters:
I20260808 18:22:14.018180 29992 leader_election.cc:288] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:14.018258 30001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:14.018458 30001 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [term 1 LEADER]: Becoming Leader. State: Replica: d0578719333b4412a38b8ef74a45f7d0, State: Running, Role: LEADER
I20260808 18:22:14.018931 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:14.019055 30001 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0578719333b4412a38b8ef74a45f7d0" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35521 } }
I20260808 18:22:14.019397 29992 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:14.020236 30001 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0578719333b4412a38b8ef74a45f7d0. Latest consensus state: current_term: 1 leader_uuid: "d0578719333b4412a38b8ef74a45f7d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0578719333b4412a38b8ef74a45f7d0" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35521 } } }
I20260808 18:22:14.020402 30001 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:14.020836 30004 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0578719333b4412a38b8ef74a45f7d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0578719333b4412a38b8ef74a45f7d0" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 35521 } } }
I20260808 18:22:14.020973 30004 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0 [sys.catalog]: This master's current role is: LEADER
W20260808 18:22:14.024148 30035 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:14.024230 30035 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:14.024288 30027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:14.025110 30027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:14.026084 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:14.022258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57343 (local address 127.27.18.254:35521)
0808 18:22:14.022370 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:14.022374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:14.022579 (+ 205us) server_negotiation.cc:407] Connection header received
0808 18:22:14.022706 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:14.022709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:14.022761 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:14.022837 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:14.023133 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:14.023695 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:14.024584 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:14.024818 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:14.024928 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:14.025688 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:14.025692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:14.025694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:14.025705 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:14.025731 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:14.025746 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:14.025824 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:14.025906 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 18:22:14.025948 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:14.030712 30027 catalog_manager.cc:1347] Generated new cluster ID: aee25390bdd6407d93401c7aa3880c1f
I20260808 18:22:14.030774 30027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:14.041038 30027 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:14.042155 30027 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:14.047293 30027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0578719333b4412a38b8ef74a45f7d0: Generated new TSK 0
I20260808 18:22:14.047948 30027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:14.054747 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 29888
2026-08-08T18:22:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:14 dist-test-slave-x4qh krb5kdc[30045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:14 dist-test-slave-x4qh krb5kdc[30045](info): set up 2 sockets
Aug 08 18:22:14 dist-test-slave-x4qh krb5kdc[30045](info): commencing operation
krb5kdc: starting...
W20260808 18:22:15.197184 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:15 dist-test-slave-x4qh krb5kdc[30045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:15.311661 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42521
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:42521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:15.412196 30064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:15.412433 30064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:15.412504 30064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:15.415879 30064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:15.415951 30064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:15.415982 30064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:15.416033 30064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:15.416067 30064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:15.420620 30064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:42521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:15.421444 30064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:15.422276 30064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:15.430063 30072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:15.430069 30080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:15.430115 30071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:15.430101 30064 server_base.cc:1034] running on GCE node
I20260808 18:22:15.430778 30064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:15.431820 30064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:15.432977 30064 hybrid_clock.cc:648] HybridClock initialized: now 1786213335432954 us; error 35 us; skew 500 ppm
Aug 08 18:22:15 dist-test-slave-x4qh krb5kdc[30045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213335, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:15.435276 30064 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:15.436158 30064 webserver.cc:460] Webserver started at http://127.27.18.254:39075/ using document root <none> and password file <none>
I20260808 18:22:15.436687 30064 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:15.436784 30064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:15.436997 30064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:15.438745 30064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "fcabfae7c7ab4421863a3110df953e90"
format_stamp: "Formatted at 2026-08-08 18:22:15 on dist-test-slave-x4qh"
server_key: "6d148d51006ae06cfb554165c0fe797b"
server_key_iv: "b5587ffcc831a02ac771fc34357a1412"
server_key_version: "encryptionkey@0"
I20260808 18:22:15.439257 30064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "fcabfae7c7ab4421863a3110df953e90"
format_stamp: "Formatted at 2026-08-08 18:22:15 on dist-test-slave-x4qh"
server_key: "6d148d51006ae06cfb554165c0fe797b"
server_key_iv: "b5587ffcc831a02ac771fc34357a1412"
server_key_version: "encryptionkey@0"
I20260808 18:22:15.442770 30064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:15.444995 30090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:15.446034 30064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 18:22:15.446165 30064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "fcabfae7c7ab4421863a3110df953e90"
format_stamp: "Formatted at 2026-08-08 18:22:15 on dist-test-slave-x4qh"
server_key: "6d148d51006ae06cfb554165c0fe797b"
server_key_iv: "b5587ffcc831a02ac771fc34357a1412"
server_key_version: "encryptionkey@0"
I20260808 18:22:15.446280 30064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:15.450165 30064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:15.452582 30064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:15.452705 30064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:15.459101 30064 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:42521
I20260808 18:22:15.459128 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:42521 every 8 connection(s)
I20260808 18:22:15.459851 30064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:15.462217 30170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:15.466704 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30064
I20260808 18:22:15.466847 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:15.467073 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90: Bootstrap starting.
I20260808 18:22:15.467103 27723 external_mini_cluster.cc:1428] Setting key 473ea77b2a40ca46d17f6b4fead45351
I20260808 18:22:15.469542 30170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:15.470389 30170 log.cc:824] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:15.472183 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:15.468315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54877 (local address 127.27.18.254:42521)
0808 18:22:15.468740 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:15.468760 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:15.468797 (+ 37us) server_negotiation.cc:407] Connection header received
0808 18:22:15.469555 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:15.469579 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:15.469925 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:15.470251 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:15.470975 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:15.470979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:15.470989 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:15.470998 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:15.471053 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:15.471069 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:15.471170 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:15.471309 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 18:22:15.471460 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":107,"threads_started":1}
I20260808 18:22:15.472484 30170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90: No bootstrap required, opened a new log
I20260808 18:22:15.474606 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:15.476089 30170 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcabfae7c7ab4421863a3110df953e90" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42521 } }
I20260808 18:22:15.476294 30170 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 18:22:15.476377 30170 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcabfae7c7ab4421863a3110df953e90, State: Initialized, Role: FOLLOWER
I20260808 18:22:15.476809 30170 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcabfae7c7ab4421863a3110df953e90" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42521 } }
I20260808 18:22:15.476956 30170 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 18:22:15.477022 30170 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 18:22:15.477136 30170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:15.478135 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:15.476602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38353 (local address 127.27.18.254:42521)
0808 18:22:15.476708 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:15.476712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:15.476930 (+ 218us) server_negotiation.cc:407] Connection header received
0808 18:22:15.477043 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:15.477046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:15.477073 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:15.477174 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:15.477796 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:15.477799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:15.477800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 18:22:15.477810 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:15.477830 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 18:22:15.477839 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:15.477903 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:15.478008 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 18:22:15.478034 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:15.478235 30170 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcabfae7c7ab4421863a3110df953e90" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42521 } }
I20260808 18:22:15.478540 30170 leader_election.cc:302] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcabfae7c7ab4421863a3110df953e90; no voters:
I20260808 18:22:15.478852 30170 leader_election.cc:288] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 18:22:15.478948 30185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:15.479099 30185 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [term 1 LEADER]: Becoming Leader. State: Replica: fcabfae7c7ab4421863a3110df953e90, State: Running, Role: LEADER
I20260808 18:22:15.479362 30185 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcabfae7c7ab4421863a3110df953e90" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42521 } }
I20260808 18:22:15.480048 30170 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:15.480906 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcabfae7c7ab4421863a3110df953e90. Latest consensus state: current_term: 1 leader_uuid: "fcabfae7c7ab4421863a3110df953e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcabfae7c7ab4421863a3110df953e90" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42521 } } }
I20260808 18:22:15.481011 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:15.481607 30197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:15.481593 30186 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcabfae7c7ab4421863a3110df953e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcabfae7c7ab4421863a3110df953e90" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 42521 } } }
I20260808 18:22:15.481809 30186 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:15.484143 30197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:15.489041 30197 catalog_manager.cc:1347] Generated new cluster ID: 0327f03efe3448f9b9bc40304b147e06
I20260808 18:22:15.489105 30197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:15.495918 30197 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:15.496146 30197 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:15.501925 30197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcabfae7c7ab4421863a3110df953e90: Generated new TSK 0
I20260808 18:22:15.502447 30197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 18:22:15.513132 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30064
2026-08-08T18:22:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1454 ms)
[----------] 18 tests from AuthTokenIssuingTest (26940 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:15 dist-test-slave-x4qh krb5kdc[30220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:15 dist-test-slave-x4qh krb5kdc[30220](info): set up 2 sockets
Aug 08 18:22:15 dist-test-slave-x4qh krb5kdc[30220](info): commencing operation
krb5kdc: starting...
W20260808 18:22:16.648145 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:16 dist-test-slave-x4qh krb5kdc[30220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:16.762298 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:45071
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:45071,127.27.18.253:34169,127.27.18.252:36613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:16.865675 30241 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:22:16.865888 30241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:16.865908 30241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:16.865940 30241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:16.869625 30241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:16.869669 30241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:16.869688 30241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:16.869704 30241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:16.869742 30241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:16.874229 30241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:45071,127.27.18.253:34169,127.27.18.252:36613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:16.875036 30241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:16.875921 30241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:16.883870 30252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:16.883880 30249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:16.884140 30241 server_base.cc:1034] running on GCE node
W20260808 18:22:16.883880 30248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:16.884721 30241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:16.885663 30241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:16.886852 30241 hybrid_clock.cc:648] HybridClock initialized: now 1786213336886826 us; error 35 us; skew 500 ppm
Aug 08 18:22:16 dist-test-slave-x4qh krb5kdc[30220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:16.895519 30241 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:16.896340 30241 webserver.cc:460] Webserver started at http://127.27.18.254:33503/ using document root <none> and password file <none>
I20260808 18:22:16.896889 30241 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:16.896979 30241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:16.897184 30241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:16.899024 30241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "0ca1868f4f1a491a9e07649d5381f687"
format_stamp: "Formatted at 2026-08-08 18:22:16 on dist-test-slave-x4qh"
server_key: "afd0f3ce2aa4ca48f88293314d598d56"
server_key_iv: "97cf76f1755987d716ada3a21af3a88b"
server_key_version: "encryptionkey@0"
I20260808 18:22:16.899595 30241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "0ca1868f4f1a491a9e07649d5381f687"
format_stamp: "Formatted at 2026-08-08 18:22:16 on dist-test-slave-x4qh"
server_key: "afd0f3ce2aa4ca48f88293314d598d56"
server_key_iv: "97cf76f1755987d716ada3a21af3a88b"
server_key_version: "encryptionkey@0"
I20260808 18:22:16.902792 30241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 18:22:16.905158 30262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:16.906302 30241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 18:22:16.906433 30241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "0ca1868f4f1a491a9e07649d5381f687"
format_stamp: "Formatted at 2026-08-08 18:22:16 on dist-test-slave-x4qh"
server_key: "afd0f3ce2aa4ca48f88293314d598d56"
server_key_iv: "97cf76f1755987d716ada3a21af3a88b"
server_key_version: "encryptionkey@0"
I20260808 18:22:16.906533 30241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:16.923966 30241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:16.926924 30241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:16.927115 30241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:16.934012 30241 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:45071
I20260808 18:22:16.934284 30343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:45071 every 8 connection(s)
I20260808 18:22:16.934900 30241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:16.937451 30344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:16.937752 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30241
I20260808 18:22:16.937897 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:16.938174 27723 external_mini_cluster.cc:1428] Setting key 85fad9e4008ee062d2a8b91b6773a77c
I20260808 18:22:16.941391 30344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.18.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.253@KRBTEST.COM" created.
Aug 08 18:22:16 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:16.955307 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:16.943507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45071 (local address 127.0.0.1:32824)
0808 18:22:16.944021 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:16.944054 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:16.944831 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:16.945501 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:16.945506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:16.945927 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:16.946142 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:16.946154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:16.947424 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:16.947427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:16.948225 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:16.948234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:16.948838 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:16.948842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:16.949022 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:16.949603 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:16.949618 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:16.950954 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:16.952715 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:16.952724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:16.952732 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:16.952967 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:16.953260 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:16.953264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:16.953266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:16.953370 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:16.953870 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:16.953875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:16.954001 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 18:22:16.954150 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 18:22:16.954493 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":186,"spinlock_wait_cycles":2432,"thread_start_us":140,"threads_started":1}
I20260808 18:22:16.955315 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:16.943506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.27.18.254:45071)
0808 18:22:16.943879 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:16.943897 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:16.944165 (+ 268us) server_negotiation.cc:407] Connection header received
0808 18:22:16.945121 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:16.945126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:16.945293 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:16.945501 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:16.946332 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:16.947265 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:16.948345 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:16.948685 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:16.948903 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:16.951098 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:16.951103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:16.951111 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:16.951129 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:16.952614 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:16.953073 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:16.953084 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:16.953085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:16.953173 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:16.953455 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:16.953460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:16.953461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:16.953757 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:16.953885 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:16.954206 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 18:22:16.954447 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":247,"thread_start_us":125,"threads_started":1}
I20260808 18:22:16.957760 30344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:16.958395 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:16.958171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52576)
0808 18:22:16.958270 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:16.958343 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:22:16.958599 30263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
I20260808 18:22:16.959430 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:16.959188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52586)
0808 18:22:16.959296 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:16.959338 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 18:22:16.959606 30344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:34169: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab.
Entry for principal kudu/127.27.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab.
I20260808 18:22:16.964207 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:34169
--webserver_interface=127.27.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab
--principal=kudu/127.27.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:45071,127.27.18.253:34169,127.27.18.252:36613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:22:17.008863 30344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } attempt: 1
I20260808 18:22:17.009652 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.009349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52596)
0808 18:22:17.009496 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.009556 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 18:22:17.010336 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.010107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52610)
0808 18:22:17.010226 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.010277 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:22:17.010509 30344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:34169: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
W20260808 18:22:17.066493 30358 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:22:17.066735 30358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:17.066757 30358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:17.066834 30358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:17.070391 30358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:17.070433 30358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:17.070451 30358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:17.070465 30358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:17.070478 30358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:17.075241 30358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:45071,127.27.18.253:34169,127.27.18.252:36613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab
--principal=kudu/127.27.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:34169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:17.076032 30358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:17.076838 30358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:22:17.078688 30344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } attempt: 2
I20260808 18:22:17.079435 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.079121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52612)
0808 18:22:17.079268 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.079331 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 18:22:17.080150 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.079905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52614)
0808 18:22:17.080038 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.080090 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 18:22:17.080328 30344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:34169: Network error: Client connection negotiation failed: client connection to 127.27.18.253:34169: connect: Connection refused (error 111)
W20260808 18:22:17.084625 30365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:17.084630 30369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:17.084630 30366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:17.084909 30358 server_base.cc:1034] running on GCE node
I20260808 18:22:17.085374 30358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:17.086242 30358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:17.087422 30358 hybrid_clock.cc:648] HybridClock initialized: now 1786213337087396 us; error 33 us; skew 500 ppm
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:17.089864 30358 init.cc:377] Logged in from keytab as kudu/127.27.18.253@KRBTEST.COM (short username kudu)
I20260808 18:22:17.090621 30358 webserver.cc:460] Webserver started at http://127.27.18.253:38203/ using document root <none> and password file <none>
I20260808 18:22:17.091109 30358 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:17.091164 30358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:17.091308 30358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:17.093060 30358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data/instance:
uuid: "c2caa48fbe9c4172907f41337f7f690a"
format_stamp: "Formatted at 2026-08-08 18:22:17 on dist-test-slave-x4qh"
server_key: "6c5ee45009b28253be7f2e0ab944c855"
server_key_iv: "ee34e81c11562878f2fd93ff70aba00b"
server_key_version: "encryptionkey@0"
I20260808 18:22:17.093500 30358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal/instance:
uuid: "c2caa48fbe9c4172907f41337f7f690a"
format_stamp: "Formatted at 2026-08-08 18:22:17 on dist-test-slave-x4qh"
server_key: "6c5ee45009b28253be7f2e0ab944c855"
server_key_iv: "ee34e81c11562878f2fd93ff70aba00b"
server_key_version: "encryptionkey@0"
I20260808 18:22:17.096525 30358 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 18:22:17.098635 30376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:17.099810 30358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:17.099922 30358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal
uuid: "c2caa48fbe9c4172907f41337f7f690a"
format_stamp: "Formatted at 2026-08-08 18:22:17 on dist-test-slave-x4qh"
server_key: "6c5ee45009b28253be7f2e0ab944c855"
server_key_iv: "ee34e81c11562878f2fd93ff70aba00b"
server_key_version: "encryptionkey@0"
I20260808 18:22:17.100006 30358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:17.117268 30358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:17.119656 30358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:17.119827 30358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:17.126070 30358 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.253:34169
I20260808 18:22:17.126322 30470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.253:34169 every 8 connection(s)
I20260808 18:22:17.126912 30358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
I20260808 18:22:17.129273 30471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:17.129709 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30358
I20260808 18:22:17.129840 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-1/wal/instance
I20260808 18:22:17.130105 27723 external_mini_cluster.cc:1428] Setting key 4674ce7a2398a87994550420936ee27f
I20260808 18:22:17.133496 30471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.18.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.252@KRBTEST.COM" created.
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:17.147707 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.135124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.27.18.254:45071)
0808 18:22:17.135234 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.135254 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.136168 (+ 914us) server_negotiation.cc:407] Connection header received
0808 18:22:17.137391 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.137394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.137441 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.137511 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.138917 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.139559 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.140772 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.141002 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.141074 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.143563 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.143566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.143568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.143591 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.144954 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.145572 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.145575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.145576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.145649 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.146031 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.146034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.146035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.146180 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.146251 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.147496 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.147598 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:22:17.148208 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.135533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45071 (local address 127.0.0.1:32832)
0808 18:22:17.135989 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.136043 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.137154 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.137708 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.137717 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.138144 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.138770 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.138791 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.139713 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.139721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.140667 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.140677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.141204 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.141207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.141342 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.141869 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.141889 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.143367 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.145072 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.145080 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.145096 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.145454 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.145746 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.145752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.145757 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.145941 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.146289 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.146296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.146610 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.146842 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.147165 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":91,"threads_started":1}
I20260808 18:22:17.149499 30471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab.
Entry for principal kudu/127.27.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab.
I20260808 18:22:17.158385 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.149930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52630)
0808 18:22:17.150036 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.150051 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.150125 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.150656 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.150659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.150843 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.150930 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.150935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.152011 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.152017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.152653 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.152663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.153273 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.153277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.153370 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.153794 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.153837 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.155119 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.156874 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.156879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.156881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.157120 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.157461 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.157464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.157465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.157566 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.158052 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.158055 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.158127 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.158208 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.158289 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 18:22:17.158381 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:36613
--webserver_interface=127.27.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab
--principal=kudu/127.27.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:45071,127.27.18.253:34169,127.27.18.252:36613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:22:17.158818 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.150094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.27.18.253:34169)
0808 18:22:17.150378 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.150386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.150401 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:17.150430 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.150433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.150547 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.150672 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.151038 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.151896 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.152760 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.153109 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.153274 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.155257 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.155266 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.155278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.155313 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.156767 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.157233 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.157238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.157240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.157352 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.157647 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.157650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.157652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.157939 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.158065 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.158325 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.158578 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":129,"threads_started":1}
I20260808 18:22:17.160372 30471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:17.161082 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.160849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32890)
0808 18:22:17.160970 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.161036 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 18:22:17.161365 30378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
I20260808 18:22:17.162194 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.162008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32906)
0808 18:22:17.162111 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.162151 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:22:17.162427 30471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:36613: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
I20260808 18:22:17.171416 30344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } attempt: 3
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:22:17.178596 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.171773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52642)
0808 18:22:17.171877 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.171887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.171946 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.172139 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.172141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.172260 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.172350 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.172354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.173232 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.173234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.173661 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.173667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.174158 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.174160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.174226 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.174616 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.174626 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.175644 (+ 1018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.177285 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.177288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.177289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.177466 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.177742 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.177744 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.177746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.177825 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.178173 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.178175 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.178238 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.178354 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.178465 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 18:22:17.178596 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.171824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.27.18.253:34169)
0808 18:22:17.171928 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.171932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.171944 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:22:17.171999 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.172002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.172035 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.172101 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.172441 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.173126 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.173750 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.174066 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.174153 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.175759 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.175764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.175766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.175789 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.177179 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.177561 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.177564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.177566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.177651 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.177916 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.177918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.177920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.178090 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.178177 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.178404 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.178494 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:22:17.179708 30344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:17.180240 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.180070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32916)
0808 18:22:17.180160 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.180197 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 18:22:17.180816 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.180617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32930)
0808 18:22:17.180735 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.180776 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 18:22:17.180941 30344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:36613: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
I20260808 18:22:17.211651 30471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } attempt: 1
I20260808 18:22:17.212093 30344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } attempt: 1
I20260808 18:22:17.212316 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.212038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32942)
0808 18:22:17.212176 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.212233 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:22:17.212635 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.212415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32944)
0808 18:22:17.212517 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.212560 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 18:22:17.213241 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.213035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32952)
0808 18:22:17.213135 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.213181 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 18:22:17.213372 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.213151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32962)
0808 18:22:17.213261 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.213315 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 18:22:17.213532 30344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:36613: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
W20260808 18:22:17.213626 30471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:36613: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
W20260808 18:22:17.264138 30480 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:22:17.264366 30480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:17.264437 30480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:17.264503 30480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:17.268067 30480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:17.268107 30480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:17.268124 30480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:17.268138 30480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:17.268150 30480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:17.272810 30480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:45071,127.27.18.253:34169,127.27.18.252:36613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab
--principal=kudu/127.27.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:36613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:17.273625 30480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:17.274487 30480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:22:17.281747 30471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } attempt: 2
W20260808 18:22:17.282277 30490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:17.282415 30491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:17.282292 30494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:17.282702 30480 server_base.cc:1034] running on GCE node
I20260808 18:22:17.282752 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.282285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32970)
0808 18:22:17.282557 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.282616 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260808 18:22:17.283313 30480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:17.283522 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.283246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32978)
0808 18:22:17.283407 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.283467 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 18:22:17.283713 30471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:36613: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
I20260808 18:22:17.284376 30480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:17.285561 30480 hybrid_clock.cc:648] HybridClock initialized: now 1786213337285544 us; error 35 us; skew 500 ppm
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:17.288030 30480 init.cc:377] Logged in from keytab as kudu/127.27.18.252@KRBTEST.COM (short username kudu)
I20260808 18:22:17.288777 30480 webserver.cc:460] Webserver started at http://127.27.18.252:40695/ using document root <none> and password file <none>
I20260808 18:22:17.288717 30344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } attempt: 2
I20260808 18:22:17.289323 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.289109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:32994)
0808 18:22:17.289221 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.289264 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:22:17.289369 30480 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:17.289448 30480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:17.289672 30480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:17.289942 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.289769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:33004)
0808 18:22:17.289860 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.289897 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 18:22:17.290135 30344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:36613: Network error: Client connection negotiation failed: client connection to 127.27.18.252:36613: connect: Connection refused (error 111)
I20260808 18:22:17.291699 30480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data/instance:
uuid: "c619047392804984862d8bdbca025e37"
format_stamp: "Formatted at 2026-08-08 18:22:17 on dist-test-slave-x4qh"
server_key: "ee0667ef7377dfe2031a6dd292e92c97"
server_key_iv: "408d2e054b3e413f8737d1a6e1a76a86"
server_key_version: "encryptionkey@0"
I20260808 18:22:17.292236 30480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal/instance:
uuid: "c619047392804984862d8bdbca025e37"
format_stamp: "Formatted at 2026-08-08 18:22:17 on dist-test-slave-x4qh"
server_key: "ee0667ef7377dfe2031a6dd292e92c97"
server_key_iv: "408d2e054b3e413f8737d1a6e1a76a86"
server_key_version: "encryptionkey@0"
I20260808 18:22:17.295742 30480 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 18:22:17.297807 30503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:17.298898 30480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:17.299029 30480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal
uuid: "c619047392804984862d8bdbca025e37"
format_stamp: "Formatted at 2026-08-08 18:22:17 on dist-test-slave-x4qh"
server_key: "ee0667ef7377dfe2031a6dd292e92c97"
server_key_iv: "408d2e054b3e413f8737d1a6e1a76a86"
server_key_version: "encryptionkey@0"
I20260808 18:22:17.299126 30480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:17.308373 30480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:17.311065 30480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:17.311240 30480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:17.317798 30480 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.252:36613
I20260808 18:22:17.318051 30599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.252:36613 every 8 connection(s)
I20260808 18:22:17.318666 30480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
I20260808 18:22:17.321106 30600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:17.324605 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30480
I20260808 18:22:17.324736 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786213058405439-27723-0/minicluster-data/master-2/wal/instance
I20260808 18:22:17.324507 30600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:17.325027 27723 external_mini_cluster.cc:1428] Setting key c42c4dc5595df5c8293047f8b8c306bd
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:22:17.336835 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.326382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.27.18.254:45071)
0808 18:22:17.326490 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.326496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.327008 (+ 512us) server_negotiation.cc:407] Connection header received
0808 18:22:17.327842 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.327845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.327889 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.327960 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.328846 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.329302 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.330341 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.330625 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.330699 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.333092 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.333102 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.333104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.333126 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.334589 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.335192 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.335195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.335196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.335263 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.335610 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.335613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.335615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.335815 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.335950 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.336532 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.336669 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 18:22:17.337625 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.326302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.27.18.252:36613)
0808 18:22:17.326765 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.326783 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.326829 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:22:17.327631 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.327654 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.327885 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.328065 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.328777 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.329675 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.330530 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.330932 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.331025 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.333455 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.333460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.333468 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.333487 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.335099 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.335717 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.335721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.335721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.335783 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.336075 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.336078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.336080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.336358 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.336524 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.336749 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.336862 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"spinlock_wait_cycles":44800,"thread_start_us":134,"threads_started":1}
I20260808 18:22:17.337630 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.326309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:45071 (local address 127.0.0.1:32848)
0808 18:22:17.326857 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.326900 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.327631 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.328032 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.328037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.328431 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.328660 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.328672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.329405 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.329408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.330221 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.330229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.330751 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.330754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.330918 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.331536 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.331551 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.332920 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.334694 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.334701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.334709 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.335099 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.335338 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.335340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.335342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.335512 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.335959 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.335963 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.336247 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.336428 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.336723 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":130,"threads_started":1}
I20260808 18:22:17.338896 30600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:17.347496 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.339423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:34169 (local address 127.0.0.1:52650)
0808 18:22:17.339532 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.339547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.339620 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.339956 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.339962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.340125 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.340234 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.340241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.341123 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.341126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.341733 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.341742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.342275 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.342279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.342377 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.342810 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.342821 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.344047 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.345942 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.345948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.345952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.346180 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.346448 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.346451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.346452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.346548 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.346990 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.346996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.347088 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.347196 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.347320 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 18:22:17.347556 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.339522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52650 (local address 127.27.18.253:34169)
0808 18:22:17.339647 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.339653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.339669 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:22:17.339711 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.339714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.339756 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.339828 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.340382 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.341024 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.341844 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.342180 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.342253 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.344178 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.344180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.344182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.344204 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.345826 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.346276 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.346278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.346279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.346357 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.346650 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.346657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.346661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.346882 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.346989 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.347263 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.347426 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 18:22:17.348165 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.339821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.27.18.254:45071)
0808 18:22:17.339931 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.339934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.339946 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:22:17.340028 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.340033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.340076 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.340155 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.340811 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.341329 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.342128 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.342387 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.342500 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.344895 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.344898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.344901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.344923 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.346463 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.346936 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.346938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.346956 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.347040 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.347341 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.347344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.347345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.347581 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.347791 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.347919 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.348026 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 18:22:17.348250 30600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:22:17.355732 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.348690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:33020)
0808 18:22:17.348833 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.348844 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.348941 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.349161 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.349164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.349341 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.349451 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.349457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.350165 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.350167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.350643 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.350652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.351048 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.351051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.351124 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.351539 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.351550 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.352675 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.354395 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.354400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.354401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.354570 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.354830 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.354834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.354836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.354905 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.355230 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.355232 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.355313 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.355453 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.355568 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 18:22:17.355743 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.348715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.27.18.252:36613)
0808 18:22:17.348806 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.348810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.348909 (+ 99us) server_negotiation.cc:407] Connection header received
0808 18:22:17.349007 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.349011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.349048 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.349116 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.349557 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.350062 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.350746 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.350950 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.351018 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.352793 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.352796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.352799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.352825 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.354286 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.354658 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.354663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.354664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.354731 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.354987 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.354990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.354991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.355142 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.355236 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.355498 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.355616 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":34}
I20260808 18:22:17.357008 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.349149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.27.18.253:34169)
0808 18:22:17.349249 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.349252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.349305 (+ 53us) server_negotiation.cc:407] Connection header received
0808 18:22:17.349430 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.349432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.349460 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.349541 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.350028 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.350488 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.351289 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.351607 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.351679 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.354099 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.354102 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.354104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.354124 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.355508 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.355939 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.355944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.355947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.356025 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.356283 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.356285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.356286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.356453 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.356664 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.356752 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.356870 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:22:17.359037 30600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: Bootstrap starting.
I20260808 18:22:17.360070 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:17.361646 30600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:17.362350 30600 log.cc:824] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:17.361469 30301 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0ca1868f4f1a491a9e07649d5381f687"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32856
I20260808 18:22:17.364113 30600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: No bootstrap required, opened a new log
I20260808 18:22:17.366652 30600 raft_consensus.cc:348] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.366874 30600 raft_consensus.cc:729] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c619047392804984862d8bdbca025e37, State: Initialized, Role: FOLLOWER
I20260808 18:22:17.367163 30600 consensus_queue.cc:260] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.367666 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.367795 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.368263 30600 sys_catalog.cc:564] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 18:22:17.372849 30626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:17.372910 30626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:17.374840 30471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } attempt: 3
I20260808 18:22:17.371726 30480 master_runner.cc:186] Error getting master registration for 127.27.18.254:45071: OK, instance_id { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" instance_seqno: 1786213336932222 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:22:17.375301 30480 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213337, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:22:17.382807 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.375214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:33022)
0808 18:22:17.375327 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.375342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.375473 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.375781 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.375784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.375945 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.376020 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.376024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.376673 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.376677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.377184 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.377191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.377640 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.377643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.377728 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.378187 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.378200 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.379649 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.381433 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.381436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.381438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.381620 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.381870 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.381872 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.381874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.381948 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.382348 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.382350 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.382441 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.382541 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.382663 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 18:22:17.382815 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.375494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.27.18.252:36613)
0808 18:22:17.375604 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.375608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.375619 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:22:17.375650 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.375652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.375686 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.375750 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.376125 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.376568 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.377307 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.377531 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.377635 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.379764 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.379767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.379769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.379789 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.381308 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.381709 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.381714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.381716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.381787 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.382050 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.382054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.382056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.382267 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.382352 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.382591 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.382683 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 18:22:17.385659 30471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: Bootstrap starting.
I20260808 18:22:17.388062 30471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:17.388783 30471 log.cc:824] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:17.389326 30344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } attempt: 3
I20260808 18:22:17.390628 30471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: No bootstrap required, opened a new log
Aug 08 18:22:17 dist-test-slave-x4qh krb5kdc[30220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213336, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:22:17.394842 30471 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.395036 30471 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2caa48fbe9c4172907f41337f7f690a, State: Initialized, Role: FOLLOWER
I20260808 18:22:17.395318 30471 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.395947 30630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.396204 30630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.396565 30471 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:17.397714 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.389683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:36613 (local address 127.0.0.1:33036)
0808 18:22:17.389826 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:17.389840 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:17.389915 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:17.390084 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:17.390086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:17.390232 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:17.390309 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.390314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.391190 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.391193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.391798 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:17.391826 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.392364 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.392367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:17.392450 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.392905 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:17.392918 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:17.394255 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:17.396123 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.396126 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.396128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.396397 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.396793 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:17.396795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:17.396797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:17.396881 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:17.397275 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:17.397280 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:17.397387 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 18:22:17.397489 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 18:22:17.397614 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 18:22:17.398169 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.389739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.27.18.252:36613)
0808 18:22:17.389827 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.389831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.389878 (+ 47us) server_negotiation.cc:407] Connection header received
0808 18:22:17.389977 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.389980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.390013 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.390069 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.390408 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.391076 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.391936 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.392268 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.392360 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.394372 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.394375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.394378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.394398 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.395926 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.396578 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.396581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.396582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.396681 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.397004 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.397007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.397009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.397184 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.397266 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.397930 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.398032 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 18:22:17.400894 30344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: Bootstrap starting.
W20260808 18:22:17.401329 30650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:17.401441 30650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:17.403079 30344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:17.403867 30344 log.cc:824] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:17.404646 30358 master_runner.cc:186] Error getting master registration for 127.27.18.254:45071: OK, instance_id { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" instance_seqno: 1786213336932222 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:22:17.407887 30358 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:22:17.409168 30344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: No bootstrap required, opened a new log
I20260808 18:22:17.411963 30344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.412170 30344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca1868f4f1a491a9e07649d5381f687, State: Initialized, Role: FOLLOWER
I20260808 18:22:17.412439 30344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.412875 30654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.413126 30654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.413543 30344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:17.413619 30301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 18:22:17.413723 30301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:17.414848 30301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
I20260808 18:22:17.415874 30301 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [CANDIDATE]: Term 1 election: Requested vote from peers c2caa48fbe9c4172907f41337f7f690a (127.27.18.253:34169), c619047392804984862d8bdbca025e37 (127.27.18.252:36613)
I20260808 18:22:17.416808 30421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca1868f4f1a491a9e07649d5381f687" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2caa48fbe9c4172907f41337f7f690a"
I20260808 18:22:17.417029 30421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:17.418246 30421 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca1868f4f1a491a9e07649d5381f687 in term 1.
I20260808 18:22:17.418901 30552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca1868f4f1a491a9e07649d5381f687" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c619047392804984862d8bdbca025e37"
I20260808 18:22:17.419100 30552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [term 0 FOLLOWER]: Advancing to term 1
W20260808 18:22:17.419368 30676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:17.419451 30676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:17.420358 30552 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca1868f4f1a491a9e07649d5381f687 in term 1.
I20260808 18:22:17.422461 30264 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca1868f4f1a491a9e07649d5381f687, c619047392804984862d8bdbca025e37; no voters:
I20260808 18:22:17.427068 30241 master_runner.cc:263] No leader master found from master 0ca1868f4f1a491a9e07649d5381f687
I20260808 18:22:17.427124 30241 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:22:17.427910 30654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:17.428133 30654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [term 1 LEADER]: Becoming Leader. State: Replica: 0ca1868f4f1a491a9e07649d5381f687, State: Running, Role: LEADER
I20260808 18:22:17.428426 30654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } }
W20260808 18:22:17.428550 30269 tablet.cc:2369] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 18:22:17.431098 30654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca1868f4f1a491a9e07649d5381f687. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.433038 30654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:17.440238 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.422885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.27.18.254:45071)
0808 18:22:17.423175 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.423179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.428794 (+ 5615us) server_negotiation.cc:407] Connection header received
0808 18:22:17.428922 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.428924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.428979 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.429049 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.431273 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.431981 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.433317 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.433614 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.433681 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.436055 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.436059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.436062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.436085 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.437683 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.438140 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.438144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.438145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.438215 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.438681 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.438684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.438685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.438862 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.438976 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.439926 (+ 950us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.440051 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260808 18:22:17.442474 30694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:17.444248 30694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:17.451694 30552 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [term 1 FOLLOWER]: Refusing update from remote peer 0ca1868f4f1a491a9e07649d5381f687: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:17.452529 30686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:17.456828 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca1868f4f1a491a9e07649d5381f687. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.456945 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.458036 30421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [term 1 FOLLOWER]: Refusing update from remote peer 0ca1868f4f1a491a9e07649d5381f687: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:17.458586 30686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:17.462332 30630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca1868f4f1a491a9e07649d5381f687. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.462459 30630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.463201 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.435628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.27.18.252:36613)
0808 18:22:17.435722 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.435726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.439648 (+ 3922us) server_negotiation.cc:407] Connection header received
0808 18:22:17.439839 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.439845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.439910 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.439960 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.443486 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.444095 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.445152 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.445438 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.445522 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.449609 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.449613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.449616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.449641 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.451278 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.459525 (+ 8247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.459529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.459530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.459627 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.459975 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.459978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.459980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.460187 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.460302 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.462932 (+ 2630us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.463048 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:22:17.466014 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:17.427454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.27.18.253:34169)
0808 18:22:17.427553 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.427557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.438624 (+ 11067us) server_negotiation.cc:407] Connection header received
0808 18:22:17.439504 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.439507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.439555 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.439612 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.444417 (+ 4805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.445072 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.447607 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:17.447944 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:17.448020 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:17.460616 (+ 12596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:17.460619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:17.460621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:17.460644 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:17.462362 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.464640 (+ 2278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.464644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.464645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.464751 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:17.465089 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:17.465094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:17.465096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:17.465302 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:17.465410 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:17.465777 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 18:22:17.465892 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 18:22:17.466748 30686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.466881 30686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:17.468776 30630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.468880 30630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.469492 30698 mvcc.cc:205] Tried to move back new op lower bound from 7316329830194384896 to 7316329830114627584. Current Snapshot: MvccSnapshot[applied={T|T < 7316329830194384896}]
I20260808 18:22:17.472308 30702 mvcc.cc:205] Tried to move back new op lower bound from 7316329830194384896 to 7316329830114627584. Current Snapshot: MvccSnapshot[applied={T|T < 7316329830194384896}]
I20260808 18:22:17.472992 30654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.473088 30654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:17.474108 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ca1868f4f1a491a9e07649d5381f687" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca1868f4f1a491a9e07649d5381f687" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 45071 } } peers { permanent_uuid: "c2caa48fbe9c4172907f41337f7f690a" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 34169 } } peers { permanent_uuid: "c619047392804984862d8bdbca025e37" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 36613 } } }
I20260808 18:22:17.474220 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:17.476994 30700 mvcc.cc:205] Tried to move back new op lower bound from 7316329830194384896 to 7316329830114627584. Current Snapshot: MvccSnapshot[applied={T|T < 7316329830194384896}]
I20260808 18:22:17.482787 30694 catalog_manager.cc:1347] Generated new cluster ID: 471e4687d10a4dc485a9287ff67bfb3e
I20260808 18:22:17.482838 30694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:17.525307 30694 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:17.526520 30694 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:17.539705 30694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ca1868f4f1a491a9e07649d5381f687: Generated new TSK 0
I20260808 18:22:17.540436 30694 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:22:17.569067 30348 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.566137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.27.18.254:45071)
0808 18:22:17.566252 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.566256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.568023 (+ 1767us) server_negotiation.cc:407] Connection header received
0808 18:22:17.568227 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.568230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.568279 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.568330 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.568957 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32880: BlockingRecv error: recv got EOF from 127.0.0.1:32880 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:22:17.570459 30604 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.566860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.27.18.252:36613)
0808 18:22:17.566969 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.566973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.569592 (+ 2619us) server_negotiation.cc:407] Connection header received
0808 18:22:17.569738 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.569741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.569779 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.569830 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.570340 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33052: BlockingRecv error: recv got EOF from 127.0.0.1:33052 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 18:22:17.571784 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.567481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.27.18.253:34169)
0808 18:22:17.567582 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.567587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.570998 (+ 3411us) server_negotiation.cc:407] Connection header received
0808 18:22:17.571129 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.571132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.571182 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.571258 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.571658 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52666: BlockingRecv error: recv got EOF from 127.0.0.1:52666 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 18:22:17.575424 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.573965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.27.18.253:34169)
0808 18:22:17.574053 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.574056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.574540 (+ 484us) server_negotiation.cc:407] Connection header received
0808 18:22:17.574740 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.574743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.574808 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.574858 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.575322 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52668: BlockingRecv error: recv got EOF from 127.0.0.1:52668 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 18:22:17.579643 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.578779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.27.18.253:34169)
0808 18:22:17.578885 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.578889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.578906 (+ 17us) server_negotiation.cc:407] Connection header received
0808 18:22:17.578949 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.578952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.578994 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.579079 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.579563 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52670: BlockingRecv error: recv got EOF from 127.0.0.1:52670 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 18:22:17.584539 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.583333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.27.18.253:34169)
0808 18:22:17.583459 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.583463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.583724 (+ 261us) server_negotiation.cc:407] Connection header received
0808 18:22:17.583854 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.583857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.583898 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.583964 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.584458 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52678: BlockingRecv error: recv got EOF from 127.0.0.1:52678 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:22:17.591460 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.590357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.27.18.253:34169)
0808 18:22:17.590487 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.590492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.590664 (+ 172us) server_negotiation.cc:407] Connection header received
0808 18:22:17.590763 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.590766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.590832 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.590939 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.591352 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52692: BlockingRecv error: recv got EOF from 127.0.0.1:52692 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 18:22:17.602496 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.601206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.27.18.253:34169)
0808 18:22:17.601312 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.601316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.601472 (+ 156us) server_negotiation.cc:407] Connection header received
0808 18:22:17.601619 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.601622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.601668 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.601716 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.602400 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52694: BlockingRecv error: recv got EOF from 127.0.0.1:52694 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 18:22:17.621551 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.620397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52700 (local address 127.27.18.253:34169)
0808 18:22:17.620539 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.620561 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.620736 (+ 175us) server_negotiation.cc:407] Connection header received
0808 18:22:17.620893 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.620896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.620968 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.621054 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.621461 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52700: BlockingRecv error: recv got EOF from 127.0.0.1:52700 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 18:22:17.656211 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.655177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.27.18.253:34169)
0808 18:22:17.655299 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.655303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.655455 (+ 152us) server_negotiation.cc:407] Connection header received
0808 18:22:17.655590 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.655594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.655635 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.655723 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.656120 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52712: BlockingRecv error: recv got EOF from 127.0.0.1:52712 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 18:22:17.722668 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.721814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.27.18.253:34169)
0808 18:22:17.721946 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.721949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.722017 (+ 68us) server_negotiation.cc:407] Connection header received
0808 18:22:17.722109 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.722112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.722148 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.722210 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.722572 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52722: BlockingRecv error: recv got EOF from 127.0.0.1:52722 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 18:22:17.853721 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:17.852668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52730 (local address 127.27.18.253:34169)
0808 18:22:17.852834 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:17.852838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:17.852929 (+ 91us) server_negotiation.cc:407] Connection header received
0808 18:22:17.853050 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:17.853052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:17.853097 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:17.853165 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:17.853579 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52730: BlockingRecv error: recv got EOF from 127.0.0.1:52730 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 18:22:18.113024 30478 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:18.111915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.27.18.253:34169)
0808 18:22:18.112047 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:18.112051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:18.112152 (+ 101us) server_negotiation.cc:407] Connection header received
0808 18:22:18.112319 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:18.112322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:18.112378 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:18.112465 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:18.112910 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52738: BlockingRecv error: recv got EOF from 127.0.0.1:52738 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 18:22:18.374092 30626 catalog_manager.cc:1259] Loaded cluster ID: 471e4687d10a4dc485a9287ff67bfb3e
I20260808 18:22:18.374162 30626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: loading cluster ID for follower catalog manager: success
I20260808 18:22:18.375670 30626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: acquiring CA information for follower catalog manager: success
I20260808 18:22:18.376760 30626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c619047392804984862d8bdbca025e37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:22:18.402562 30650 catalog_manager.cc:1259] Loaded cluster ID: 471e4687d10a4dc485a9287ff67bfb3e
I20260808 18:22:18.402614 30650 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: loading cluster ID for follower catalog manager: success
I20260808 18:22:18.404204 30650 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: acquiring CA information for follower catalog manager: success
I20260808 18:22:18.405227 30650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2caa48fbe9c4172907f41337f7f690a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:22:18.630883 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:18.627038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.27.18.253:34169)
0808 18:22:18.627281 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:18.627286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:18.627412 (+ 126us) server_negotiation.cc:407] Connection header received
0808 18:22:18.627620 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:18.627622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:18.627745 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:18.627866 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:18.628134 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.628631 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.629448 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.629674 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.629776 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:18.630089 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:18.630504 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:18.630628 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:18.630684 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 18:22:18.630735 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
I20260808 18:22:18.636674 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:18.633751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.27.18.252:36613)
0808 18:22:18.633961 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:18.633964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:18.633977 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:18.634065 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:18.634067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:18.634163 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:18.634253 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:18.634432 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.634878 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.635612 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.635806 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.635878 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:18.636092 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:18.636394 (+ 302us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:18.636469 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:18.636530 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 18:22:18.636571 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260808 18:22:18.643234 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:18.639132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.27.18.253:34169)
0808 18:22:18.639227 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:18.639230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:18.639720 (+ 490us) server_negotiation.cc:407] Connection header received
0808 18:22:18.639928 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:18.639931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:18.639978 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:18.640023 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:18.640483 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.640922 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.642166 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.642490 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.642560 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:18.642798 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:18.642864 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:18.642937 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:18.643056 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 18:22:18.643101 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 18:22:18.643532 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:18.639140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.27.18.254:45071)
0808 18:22:18.639729 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:18.639733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:18.639748 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:18.639849 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:18.639852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:18.639991 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:18.640148 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:18.640468 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.641206 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.642007 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.642324 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.642413 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:18.642681 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:18.642905 (+ 224us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:18.643273 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:18.643316 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 18:22:18.643391 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":53,"threads_started":1}
I20260808 18:22:18.643970 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:18.639614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.27.18.252:36613)
0808 18:22:18.639731 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:18.639736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:18.639834 (+ 98us) server_negotiation.cc:407] Connection header received
0808 18:22:18.639935 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:18.639938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:18.639976 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:18.640047 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:18.640354 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.641054 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.642647 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:18.642931 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:18.643025 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:18.643250 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:18.643443 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:18.643507 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:18.643800 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 18:22:18.643856 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 18:22:18.646468 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30241
I20260808 18:22:18.663708 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30358
I20260808 18:22:18.675221 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30480
2026-08-08T18:22:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3160 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 18:22:18 dist-test-slave-x4qh krb5kdc[30841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 18:22:18 dist-test-slave-x4qh krb5kdc[30841](info): set up 2 sockets
Aug 08 18:22:18 dist-test-slave-x4qh krb5kdc[30841](info): commencing operation
krb5kdc: starting...
W20260808 18:22:19.830183 27723 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/test-user.keytab.
Aug 08 18:22:19 dist-test-slave-x4qh krb5kdc[30841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T18:22:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T18:22:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
Entry for principal kudu/127.27.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab.
I20260808 18:22:19.946431 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:46419
--webserver_interface=127.27.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:46419,127.27.18.253:32791,127.27.18.252:41085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 18:22:20.047421 30859 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:22:20.047638 30859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:20.047660 30859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:20.047744 30859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:20.051276 30859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:20.051321 30859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:20.051337 30859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:20.051352 30859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:20.051426 30859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:20.055768 30859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:46419,127.27.18.253:32791,127.27.18.252:41085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.254.keytab
--principal=kudu/127.27.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.254:46419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:20.056545 30859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:20.057415 30859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:20.065340 30871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:20.065438 30859 server_base.cc:1034] running on GCE node
W20260808 18:22:20.065367 30866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:20.065354 30865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:20.066104 30859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:20.067016 30859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:20.068166 30859 hybrid_clock.cc:648] HybridClock initialized: now 1786213340068148 us; error 28 us; skew 500 ppm
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:20.070506 30859 init.cc:377] Logged in from keytab as kudu/127.27.18.254@KRBTEST.COM (short username kudu)
I20260808 18:22:20.071317 30859 webserver.cc:460] Webserver started at http://127.27.18.254:38857/ using document root <none> and password file <none>
I20260808 18:22:20.071815 30859 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:20.071868 30859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:20.072072 30859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:20.073786 30859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data/instance:
uuid: "8b48ebffdd6743f6a092a4443f3b9ba6"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "366d9fb42012903aa1d74e059d9dcd3f"
server_key_iv: "a625c931a21d84812b00a85b7d1d235f"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.074318 30859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal/instance:
uuid: "8b48ebffdd6743f6a092a4443f3b9ba6"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "366d9fb42012903aa1d74e059d9dcd3f"
server_key_iv: "a625c931a21d84812b00a85b7d1d235f"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.077459 30859 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 18:22:20.079465 30882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:20.080446 30859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:20.080591 30859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal
uuid: "8b48ebffdd6743f6a092a4443f3b9ba6"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "366d9fb42012903aa1d74e059d9dcd3f"
server_key_iv: "a625c931a21d84812b00a85b7d1d235f"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.080708 30859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:20.092056 30859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:20.094312 30859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:20.094492 30859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:20.101245 30859 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.254:46419
I20260808 18:22:20.101547 30965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.254:46419 every 8 connection(s)
I20260808 18:22:20.102111 30859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/data/info.pb
I20260808 18:22:20.104626 30967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:20.107898 30967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:20.112015 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30859
I20260808 18:22:20.112133 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-0/wal/instance
I20260808 18:22:20.112429 27723 external_mini_cluster.cc:1428] Setting key 1c47b59e0a38ba108bfd642fb7b7e715
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.18.253@KRBTEST.COM; defaulting to no policy
I20260808 18:22:20.122135 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.109682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46419 (local address 127.0.0.1:58492)
0808 18:22:20.110095 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.110130 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.110870 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.111531 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.111535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.111861 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.112197 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.112216 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.113296 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.113302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.114458 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.114475 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.115326 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.115328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.115488 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.115978 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.115993 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.117380 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.119575 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.119587 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.119597 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.119821 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.120215 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.120217 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.120218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.120327 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.121023 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.121027 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.121127 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.121244 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.121444 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260808 18:22:20.122140 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.110348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.27.18.254:46419)
0808 18:22:20.110614 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.110622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.110653 (+ 31us) server_negotiation.cc:407] Connection header received
0808 18:22:20.111113 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.111117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.111276 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.111505 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.112365 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.113172 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.114575 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.115178 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.115342 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.117532 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.117540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.117551 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.117579 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.119450 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.119962 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.119969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.119971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.120113 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.120428 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.120432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.120448 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.120917 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.121057 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.121377 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.121517 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.253@KRBTEST.COM" created.
I20260808 18:22:20.124544 30967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:20.125264 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.125023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:37306)
0808 18:22:20.125132 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.125202 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:22:20.125533 30884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
I20260808 18:22:20.126442 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.126155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:37322)
0808 18:22:20.126284 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.126353 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 18:22:20.126623 30967 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:32791: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab.
Entry for principal kudu/127.27.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab.
I20260808 18:22:20.138509 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:32791
--webserver_interface=127.27.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab
--principal=kudu/127.27.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:46419,127.27.18.253:32791,127.27.18.252:41085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:22:20.175853 30967 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } attempt: 1
I20260808 18:22:20.176618 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.176369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:37326)
0808 18:22:20.176502 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.176555 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:22:20.177325 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.177112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:37338)
0808 18:22:20.177227 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.177277 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 18:22:20.177449 30967 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:32791: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
W20260808 18:22:20.245190 30975 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:22:20.245464 30975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:20.245530 30975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:20.245594 30975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 18:22:20.245605 30967 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } attempt: 2
I20260808 18:22:20.246321 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.246057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:37350)
0808 18:22:20.246195 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.246249 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 18:22:20.246996 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.246766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:37356)
0808 18:22:20.246871 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.246913 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 18:22:20.247148 30967 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.253:32791: Network error: Client connection negotiation failed: client connection to 127.27.18.253:32791: connect: Connection refused (error 111)
W20260808 18:22:20.249786 30975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:20.249853 30975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:20.249913 30975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:20.249944 30975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:20.249975 30975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:20.254662 30975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:46419,127.27.18.253:32791,127.27.18.252:41085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.253.keytab
--principal=kudu/127.27.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.253:32791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:20.255594 30975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:20.256456 30975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 18:22:20.264338 30981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:20.264374 30987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:20.264374 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:20.264613 30975 server_base.cc:1034] running on GCE node
I20260808 18:22:20.265091 30975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:20.266036 30975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:20.267207 30975 hybrid_clock.cc:648] HybridClock initialized: now 1786213340267189 us; error 26 us; skew 500 ppm
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:20.269821 30975 init.cc:377] Logged in from keytab as kudu/127.27.18.253@KRBTEST.COM (short username kudu)
I20260808 18:22:20.270601 30975 webserver.cc:460] Webserver started at http://127.27.18.253:45257/ using document root <none> and password file <none>
I20260808 18:22:20.271083 30975 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:20.271164 30975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:20.271361 30975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:20.273262 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data/instance:
uuid: "3dbb9acb276e4a68897a90705de6c21b"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "e98ca8672e00fd5f8df82b35ba8ba420"
server_key_iv: "208e4c43d31c777079b8e061a83c13d7"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.273838 30975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal/instance:
uuid: "3dbb9acb276e4a68897a90705de6c21b"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "e98ca8672e00fd5f8df82b35ba8ba420"
server_key_iv: "208e4c43d31c777079b8e061a83c13d7"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.277177 30975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 18:22:20.279273 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:20.280351 30975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 18:22:20.280483 30975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal
uuid: "3dbb9acb276e4a68897a90705de6c21b"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "e98ca8672e00fd5f8df82b35ba8ba420"
server_key_iv: "208e4c43d31c777079b8e061a83c13d7"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.280583 30975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:20.290552 30975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:20.293021 30975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:20.293211 30975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:20.299925 30975 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.253:32791
I20260808 18:22:20.300180 31083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.253:32791 every 8 connection(s)
I20260808 18:22:20.300805 30975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/data/info.pb
I20260808 18:22:20.303261 31084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:20.304526 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 30975
I20260808 18:22:20.304623 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-1/wal/instance
I20260808 18:22:20.304890 27723 external_mini_cluster.cc:1428] Setting key c3a6824d042ad775a7d2011f90a18e0a
I20260808 18:22:20.306751 31084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.18.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.18.252@KRBTEST.COM" created.
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:20.320500 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.308462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49942 (local address 127.27.18.254:46419)
0808 18:22:20.308578 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.308581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.309387 (+ 806us) server_negotiation.cc:407] Connection header received
0808 18:22:20.310211 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.310214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.310276 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.310371 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.311415 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.311923 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.313081 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.313356 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.313472 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.316881 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.316885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.316887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.316920 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.318359 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.318877 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.318881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.318883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.318966 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.319338 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.319342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.319344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.319578 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.319698 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.320235 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.320350 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 18:22:20.321142 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.308821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46419 (local address 127.0.0.1:49942)
0808 18:22:20.309253 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.309289 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.310019 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.310521 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.310527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.310841 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.311240 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.311251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.312050 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.312053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.312960 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.312969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.313507 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.313511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.313669 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.314129 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.314146 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.316730 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.318479 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.318484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.318496 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.318767 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.319065 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.319068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.319070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.319248 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.319698 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.319704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.320004 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.320176 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.320447 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":124,"threads_started":1}
I20260808 18:22:20.322552 31084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab.
Entry for principal kudu/127.27.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab.
I20260808 18:22:20.331475 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.322995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:52848)
0808 18:22:20.323099 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.323113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.323188 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.323838 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.323841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.323955 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.324035 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.324040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.324981 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.324983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.325571 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.325597 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.326300 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.326302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.326421 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.326783 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.326793 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.328142 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.329915 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.329917 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.329918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.330126 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.330440 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.330445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.330447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.330526 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.331032 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.331034 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.331121 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.331215 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.331303 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 18:22:20.331602 27723 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
/tmp/dist-test-taskPL43Mv/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:41085
--webserver_interface=127.27.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab
--principal=kudu/127.27.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.18.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.18.254:46419,127.27.18.253:32791,127.27.18.252:41085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 18:22:20.331764 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.323085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.27.18.253:32791)
0808 18:22:20.323432 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.323452 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.323477 (+ 25us) server_negotiation.cc:407] Connection header received
0808 18:22:20.323523 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.323531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.323700 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.323823 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.324146 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.324881 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.325744 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.326185 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.326260 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.328274 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.328283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.328310 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.328332 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.329792 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.330256 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.330262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.330264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.330345 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.330616 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.330621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.330623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.330896 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.331046 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.331328 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.331553 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":122,"threads_started":1}
I20260808 18:22:20.333454 31084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:20.334175 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.333862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51776)
0808 18:22:20.333996 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.334082 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 18:22:20.334372 31002 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
I20260808 18:22:20.335245 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.334951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51786)
0808 18:22:20.335105 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.335176 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 18:22:20.335402 31084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:41085: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
I20260808 18:22:20.338302 30967 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } attempt: 3
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:22:20.346235 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.338707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:52854)
0808 18:22:20.338820 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.338833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.338914 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.339135 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.339138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.339290 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.339433 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.339441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.340086 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.340090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.340982 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.340991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.341434 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.341437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.341540 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.341957 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.341968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.343225 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.344974 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.344976 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.344978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.345158 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.345429 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.345431 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.345432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.345504 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.345878 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.345880 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.345954 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.346044 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.346133 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 18:22:20.346339 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.338760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.27.18.253:32791)
0808 18:22:20.338850 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.338855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.338868 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:20.338999 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.339002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.339037 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.339120 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.339544 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.339984 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.341098 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.341329 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.341429 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.343357 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.343361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.343363 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.343404 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.344860 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.345245 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.345248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.345250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.345329 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.345595 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.345598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.345600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.345787 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.345879 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.346110 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.346215 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 18:22:20.347277 30967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:20.347930 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.347731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51796)
0808 18:22:20.347838 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.347881 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 18:22:20.348604 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.348399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51810)
0808 18:22:20.348526 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.348563 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 18:22:20.348739 30967 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:41085: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
I20260808 18:22:20.379892 30967 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } attempt: 1
I20260808 18:22:20.380590 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.380364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51816)
0808 18:22:20.380484 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.380533 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 18:22:20.381311 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.381101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51830)
0808 18:22:20.381212 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.381262 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 18:22:20.381433 30967 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:41085: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
I20260808 18:22:20.384688 31084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } attempt: 1
I20260808 18:22:20.385298 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.385084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51846)
0808 18:22:20.385201 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.385245 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 18:22:20.385946 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.385730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51848)
0808 18:22:20.385856 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.385903 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 18:22:20.386106 31084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:41085: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
W20260808 18:22:20.438429 31095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 18:22:20.438690 31095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 18:22:20.438747 31095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 18:22:20.438814 31095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 18:22:20.442376 31095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 18:22:20.442449 31095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 18:22:20.442492 31095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 18:22:20.442543 31095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 18:22:20.442576 31095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 18:22:20.447237 31095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.18.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.18.254:46419,127.27.18.253:32791,127.27.18.252:41085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/krb5kdc/kudu_127.27.18.252.keytab
--principal=kudu/127.27.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.18.252:41085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 17:46:08 UTC on ccycloud.quasar-vogpfy.root.comops.site
build id 4575377
I20260808 18:22:20.448109 31095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 18:22:20.448938 31095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 18:22:20.454277 31084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } attempt: 2
I20260808 18:22:20.455024 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.454711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51850)
0808 18:22:20.454871 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.454948 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 18:22:20.456084 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.455799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51860)
0808 18:22:20.455957 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.456019 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 18:22:20.456238 31084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:41085: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
W20260808 18:22:20.456574 31109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:20.456593 30967 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } attempt: 2
W20260808 18:22:20.456557 31107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 18:22:20.456574 31105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 18:22:20.457244 31095 server_base.cc:1034] running on GCE node
I20260808 18:22:20.457283 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.456976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51870)
0808 18:22:20.457075 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.457129 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 18:22:20.457733 31095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 18:22:20.458015 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.457807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51872)
0808 18:22:20.457912 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.457956 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 18:22:20.458163 30967 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.18.252:41085: Network error: Client connection negotiation failed: client connection to 127.27.18.252:41085: connect: Connection refused (error 111)
I20260808 18:22:20.458782 31095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 18:22:20.459964 31095 hybrid_clock.cc:648] HybridClock initialized: now 1786213340459942 us; error 42 us; skew 500 ppm
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 18:22:20.462536 31095 init.cc:377] Logged in from keytab as kudu/127.27.18.252@KRBTEST.COM (short username kudu)
I20260808 18:22:20.463295 31095 webserver.cc:460] Webserver started at http://127.27.18.252:38953/ using document root <none> and password file <none>
I20260808 18:22:20.463825 31095 fs_manager.cc:359] Metadata directory not provided
I20260808 18:22:20.463915 31095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 18:22:20.464133 31095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 18:22:20.465930 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data/instance:
uuid: "8517eae9daf6479db25be05e4a7283ee"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "1141b51b327817708c757dfee3a8ff93"
server_key_iv: "70edea6b72e2a1caec0d19901f6bfaa4"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.466459 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal/instance:
uuid: "8517eae9daf6479db25be05e4a7283ee"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "1141b51b327817708c757dfee3a8ff93"
server_key_iv: "70edea6b72e2a1caec0d19901f6bfaa4"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.469883 31095 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 18:22:20.472175 31123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 18:22:20.473250 31095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 18:22:20.473410 31095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data,/tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal
uuid: "8517eae9daf6479db25be05e4a7283ee"
format_stamp: "Formatted at 2026-08-08 18:22:20 on dist-test-slave-x4qh"
server_key: "1141b51b327817708c757dfee3a8ff93"
server_key_iv: "70edea6b72e2a1caec0d19901f6bfaa4"
server_key_version: "encryptionkey@0"
I20260808 18:22:20.473515 31095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 18:22:20.488091 31095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 18:22:20.490873 31095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 18:22:20.491061 31095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 18:22:20.497098 31095 rpc_server.cc:307] RPC server started. Bound to: 127.27.18.252:41085
I20260808 18:22:20.497360 31203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.18.252:41085 every 8 connection(s)
I20260808 18:22:20.497974 31095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/data/info.pb
I20260808 18:22:20.500527 31204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 18:22:20.503938 31204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:20.507552 27723 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu as pid 31095
I20260808 18:22:20.507687 27723 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPL43Mv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786213058405439-27723-0/minicluster-data/master-2/wal/instance
I20260808 18:22:20.507964 27723 external_mini_cluster.cc:1428] Setting key 3b6b9f3118523d5aa65f57d4c982d5b9
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:22:20.517261 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.505527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.27.18.254:46419)
0808 18:22:20.505667 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.505674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.506364 (+ 690us) server_negotiation.cc:407] Connection header received
0808 18:22:20.507240 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.507243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.507336 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.507458 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.508781 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.509561 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.510612 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.510837 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.510943 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.513405 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.513408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.513410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.513436 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.515029 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.515662 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.515665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.515666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.515745 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.516114 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.516116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.516117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.516301 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.516381 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.516985 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.517105 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 18:22:20.517966 31206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.505854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.254:46419 (local address 127.0.0.1:49946)
0808 18:22:20.506233 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.506270 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.507042 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.507613 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.507619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.507994 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.508593 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.508614 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.509665 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.509672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.510475 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.510485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.510992 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.510996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.511168 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.511694 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.511709 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.513260 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.515128 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.515135 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.515145 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.515549 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.515839 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.515845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.515850 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.516024 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.516407 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.516413 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.516688 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.516916 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.517140 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
I20260808 18:22:20.518935 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.508573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.27.18.252:41085)
0808 18:22:20.508993 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.509007 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.509053 (+ 46us) server_negotiation.cc:407] Connection header received
0808 18:22:20.509089 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.509094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.509291 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.509435 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.510117 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.510801 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.511822 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.512150 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.512269 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.515057 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.515062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.515070 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.515109 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.516831 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.517307 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.517313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.517315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.517400 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.517708 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.517711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.517712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.517926 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.518136 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.518598 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.518733 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"spinlock_wait_cycles":3328,"thread_start_us":137,"threads_started":1}
I20260808 18:22:20.519268 31204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.254@KRBTEST.COM
I20260808 18:22:20.527684 31206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.519790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.253:32791 (local address 127.0.0.1:52858)
0808 18:22:20.519901 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.519912 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.519970 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.520472 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.520475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.520628 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.520724 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.520729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.521733 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.521735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.522192 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.522199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.522671 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.522674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.522770 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.523212 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.523225 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.524412 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.526131 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.526134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.526136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.526349 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.526681 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.526686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.526691 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.526792 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.527202 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.527205 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.527299 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.527458 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.527565 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 18:22:20.527830 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.519988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.27.18.253:32791)
0808 18:22:20.520118 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.520122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.520133 (+ 11us) server_negotiation.cc:407] Connection header received
0808 18:22:20.520166 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.520168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.520240 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.520327 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.520841 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.521613 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.522314 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.522539 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.522670 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.524558 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.524564 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.524570 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.524607 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.526025 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.526451 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.526455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.526459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.526542 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.526896 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.526899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.526900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.527070 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.527148 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.527539 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.527653 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 18:22:20.528640 31204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } has no permanent_uuid. Determining permanent_uuid...
I20260808 18:22:20.529495 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.521144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.27.18.254:46419)
0808 18:22:20.521237 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.521241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.521261 (+ 20us) server_negotiation.cc:407] Connection header received
0808 18:22:20.521336 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.521338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.521370 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.521430 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.522093 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.522535 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.523306 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.523591 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.523710 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.526248 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.526251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.526252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.526273 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.527962 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.528361 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.528365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.528366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.528445 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.528764 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.528768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.528769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.528927 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.529140 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.529255 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.529354 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.252@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.18.253@KRBTEST.COM
I20260808 18:22:20.536515 31206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.529052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51892)
0808 18:22:20.529152 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.529164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.529229 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.529546 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.529549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.529707 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.529780 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.529785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.530622 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.530625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.531087 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.531093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.531731 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.531734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.531823 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.532201 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.532214 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.533313 (+ 1099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.535164 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.535167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.535168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.535374 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.535647 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.535650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.535652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.535743 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.536133 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.536137 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.536236 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.536334 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.536432 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 18:22:20.536652 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.529186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.27.18.252:41085)
0808 18:22:20.529278 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.529282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.529292 (+ 10us) server_negotiation.cc:407] Connection header received
0808 18:22:20.529325 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.529327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.529361 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.529433 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.529886 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.530512 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.531198 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.531610 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.531716 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.533444 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.533448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.533450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.533472 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.535049 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.535482 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.535485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.535486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.535551 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.535832 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.535835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.535836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.536037 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.536122 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.536410 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.536511 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 18:22:20.538981 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.530583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52864 (local address 127.27.18.253:32791)
0808 18:22:20.530704 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.530708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.530720 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:22:20.530794 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.530798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.530832 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.530921 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.531625 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.532298 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.533023 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.533300 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.533407 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.535851 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.535854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.535856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.535880 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.537518 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.537920 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.537924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.537925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.538009 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.538300 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.538303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.538305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.538454 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.538691 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.538759 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.538854 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 18:22:20.539481 31204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: Bootstrap starting.
I20260808 18:22:20.541726 31204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:20.542166 27723 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 18:22:20.542515 31204 log.cc:824] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:20.544211 31204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: No bootstrap required, opened a new log
I20260808 18:22:20.543296 30925 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49954
I20260808 18:22:20.546797 31204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.547008 31204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8517eae9daf6479db25be05e4a7283ee, State: Initialized, Role: FOLLOWER
I20260808 18:22:20.547297 31204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.547408 31084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } attempt: 3
I20260808 18:22:20.547845 31212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.547977 31212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.548620 31204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.253@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
W20260808 18:22:20.553736 31231 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:20.553846 31231 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:20.552081 31095 master_runner.cc:186] Error getting master registration for 127.27.18.254:46419: OK, instance_id { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" instance_seqno: 1786213340099292 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:22:20.555769 31095 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:22:20.556098 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.547804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51908)
0808 18:22:20.547951 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.547964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.548030 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.548370 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.548372 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.548520 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.548602 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.548609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.549498 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.549504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.550338 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.550349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.550925 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.550928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.551030 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.551434 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.551447 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.552706 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.554457 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.554462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.554464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.554693 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.554970 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.554973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.554974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.555099 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.555594 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.555598 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.555709 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.555825 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.555951 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 18:22:20.556123 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.547870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.27.18.252:41085)
0808 18:22:20.548167 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.548170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.548182 (+ 12us) server_negotiation.cc:407] Connection header received
0808 18:22:20.548225 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.548228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.548259 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.548337 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.548725 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.549357 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.550496 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.550796 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.550907 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.552854 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.552857 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.552859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.552881 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.554328 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.554802 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.554805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.554807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.554883 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.555190 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.555193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.555195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.555472 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.555590 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.555909 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.556005 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260808 18:22:20.557323 30967 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } attempt: 3
I20260808 18:22:20.559033 31084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: Bootstrap starting.
I20260808 18:22:20.561446 31084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: Neither blocks nor log segments found. Creating new log.
Aug 08 18:22:20 dist-test-slave-x4qh krb5kdc[30841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786213340, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.18.254@KRBTEST.COM for kudu/127.27.18.252@KRBTEST.COM
I20260808 18:22:20.562162 31084 log.cc:824] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:20.564118 31084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: No bootstrap required, opened a new log
I20260808 18:22:20.567013 31084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.567205 31084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbb9acb276e4a68897a90705de6c21b, State: Initialized, Role: FOLLOWER
I20260808 18:22:20.567530 31084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.567962 31242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.568091 31242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.568614 31084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:20.573175 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.557658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.18.252:41085 (local address 127.0.0.1:51922)
0808 18:22:20.557757 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 18:22:20.557773 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 18:22:20.557847 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 18:22:20.558140 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 18:22:20.558143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 18:22:20.558317 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 18:22:20.558404 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.558410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.559275 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.559278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.559934 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 18:22:20.559947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.560557 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.560560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 18:22:20.560646 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.561062 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 18:22:20.561076 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 18:22:20.562581 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 18:22:20.571625 (+ 9044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.571628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.571630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.571870 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.572194 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 18:22:20.572197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 18:22:20.572198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 18:22:20.572279 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 18:22:20.572690 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 18:22:20.572693 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 18:22:20.572776 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 18:22:20.572874 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 18:22:20.573002 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 18:22:20.573691 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.557730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.27.18.252:41085)
0808 18:22:20.557819 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.557823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.557836 (+ 13us) server_negotiation.cc:407] Connection header received
0808 18:22:20.557919 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.557922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.557957 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.558030 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.558528 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.559164 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.560070 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.560372 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.560458 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.562713 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.562716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.562719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.562740 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.571490 (+ 8750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.572004 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.572007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.572008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.572098 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.572386 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.572389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.572391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.572565 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.572648 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.573481 (+ 833us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.573577 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 18:22:20.573719 31259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:20.573777 31259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:20.571847 30975 master_runner.cc:186] Error getting master registration for 127.27.18.254:46419: OK, instance_id { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" instance_seqno: 1786213340099292 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 18:22:20.574692 30975 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 18:22:20.576195 30967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: Bootstrap starting.
I20260808 18:22:20.578502 30967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: Neither blocks nor log segments found. Creating new log.
I20260808 18:22:20.579212 30967 log.cc:824] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: Log is configured to *not* fsync() on all Append() calls
I20260808 18:22:20.581540 30967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: No bootstrap required, opened a new log
I20260808 18:22:20.585767 30967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.586040 30967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b48ebffdd6743f6a092a4443f3b9ba6, State: Initialized, Role: FOLLOWER
I20260808 18:22:20.586412 30967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.586925 31263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.587131 31263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.588136 30925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 18:22:20.588265 30925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:20.589234 30925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
I20260808 18:22:20.589990 30925 leader_election.cc:288] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [CANDIDATE]: Term 1 election: Requested vote from peers 3dbb9acb276e4a68897a90705de6c21b (127.27.18.253:32791), 8517eae9daf6479db25be05e4a7283ee (127.27.18.252:41085)
I20260808 18:22:20.590642 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8517eae9daf6479db25be05e4a7283ee"
I20260808 18:22:20.590864 31169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:20.591602 31046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3dbb9acb276e4a68897a90705de6c21b"
I20260808 18:22:20.591774 31046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [term 0 FOLLOWER]: Advancing to term 1
I20260808 18:22:20.591984 31169 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b48ebffdd6743f6a092a4443f3b9ba6 in term 1.
I20260808 18:22:20.592373 30884 leader_election.cc:302] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8517eae9daf6479db25be05e4a7283ee, 8b48ebffdd6743f6a092a4443f3b9ba6; no voters:
I20260808 18:22:20.588071 30967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 18:22:20.592978 31046 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b48ebffdd6743f6a092a4443f3b9ba6 in term 1.
I20260808 18:22:20.594473 31263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 18:22:20.595350 31263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [term 1 LEADER]: Becoming Leader. State: Replica: 8b48ebffdd6743f6a092a4443f3b9ba6, State: Running, Role: LEADER
I20260808 18:22:20.595636 31263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } }
W20260808 18:22:20.595829 30891 tablet.cc:2369] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 18:22:20.601672 31288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 18:22:20.601755 31288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 18:22:20.606599 31292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b48ebffdd6743f6a092a4443f3b9ba6. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.606694 31292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:20.611917 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.595002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.27.18.253:32791)
0808 18:22:20.595098 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.595102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.596429 (+ 1327us) server_negotiation.cc:407] Connection header received
0808 18:22:20.596568 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.596570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.596609 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.596652 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.597747 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.598461 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.602976 (+ 4515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.603270 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.603341 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.606170 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.606174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.606177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.606201 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.607907 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.610537 (+ 2630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.610541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.610542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.610638 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.610941 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.610944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.610945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.611166 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.611284 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.611660 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.611772 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 18:22:20.613652 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.594333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.27.18.254:46419)
0808 18:22:20.594421 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.594425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.596146 (+ 1721us) server_negotiation.cc:407] Connection header received
0808 18:22:20.596259 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.596261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.596299 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.596378 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.597240 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.597991 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.607925 (+ 9934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.608260 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.608366 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.610215 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.610218 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.610221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.610246 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.611904 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.612380 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.612383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.612385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.612459 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.612773 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.612777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.612778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.612971 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.613057 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.613413 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.613518 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 18:22:20.615427 31297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 18:22:20.616205 31297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 18:22:20.617928 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:20.595695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.27.18.252:41085)
0808 18:22:20.595885 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.595889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.597676 (+ 1787us) server_negotiation.cc:407] Connection header received
0808 18:22:20.599578 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.599581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.599638 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.599704 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.601054 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.601768 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.605100 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:20.605446 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:20.605516 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:20.613446 (+ 7930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 18:22:20.613450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 18:22:20.613453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 18:22:20.613478 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 18:22:20.615046 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.616203 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.616206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.616208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.616290 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 18:22:20.617044 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 18:22:20.617048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 18:22:20.617049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 18:22:20.617257 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 18:22:20.617365 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:20.617684 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 18:22:20.617791 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260808 18:22:20.622164 31169 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [term 1 FOLLOWER]: Refusing update from remote peer 8b48ebffdd6743f6a092a4443f3b9ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:20.622920 31263 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:20.624375 31046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [term 1 FOLLOWER]: Refusing update from remote peer 8b48ebffdd6743f6a092a4443f3b9ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 18:22:20.626760 31292 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 18:22:20.630043 31212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b48ebffdd6743f6a092a4443f3b9ba6. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.630151 31212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.630133 31242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b48ebffdd6743f6a092a4443f3b9ba6. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.630261 31242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.632325 31263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.632431 31263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:20.632696 31242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.632786 31242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.634454 31298 mvcc.cc:205] Tried to move back new op lower bound from 7316329843182534656 to 7316329843083149312. Current Snapshot: MvccSnapshot[applied={T|T < 7316329843182534656}]
I20260808 18:22:20.639055 31263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.639186 31263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6 [sys.catalog]: This master's current role is: LEADER
I20260808 18:22:20.640892 31303 mvcc.cc:205] Tried to move back new op lower bound from 7316329843182534656 to 7316329843083149312. Current Snapshot: MvccSnapshot[applied={T|T < 7316329843182534656}]
I20260808 18:22:20.641840 31212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b48ebffdd6743f6a092a4443f3b9ba6" member_type: VOTER last_known_addr { host: "127.27.18.254" port: 46419 } } peers { permanent_uuid: "3dbb9acb276e4a68897a90705de6c21b" member_type: VOTER last_known_addr { host: "127.27.18.253" port: 32791 } } peers { permanent_uuid: "8517eae9daf6479db25be05e4a7283ee" member_type: VOTER last_known_addr { host: "127.27.18.252" port: 41085 } } }
I20260808 18:22:20.641956 31212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee [sys.catalog]: This master's current role is: FOLLOWER
I20260808 18:22:20.642663 31301 mvcc.cc:205] Tried to move back new op lower bound from 7316329843182534656 to 7316329843083149312. Current Snapshot: MvccSnapshot[applied={T|T < 7316329843182534656}]
I20260808 18:22:20.643894 31297 catalog_manager.cc:1347] Generated new cluster ID: 557676453e254f36975cc02df653836a
I20260808 18:22:20.643954 31297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 18:22:20.661253 31297 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 18:22:20.662330 31297 catalog_manager.cc:1504] Loading token signing keys...
I20260808 18:22:20.676925 31297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b48ebffdd6743f6a092a4443f3b9ba6: Generated new TSK 0
I20260808 18:22:20.679415 31297 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 18:22:20.714182 30971 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.711262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.27.18.254:46419)
0808 18:22:20.711403 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.711408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.713099 (+ 1691us) server_negotiation.cc:407] Connection header received
0808 18:22:20.713278 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.713281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.713342 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.713396 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.714030 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49974: BlockingRecv error: recv got EOF from 127.0.0.1:49974 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 18:22:20.715561 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.711997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52892 (local address 127.27.18.253:32791)
0808 18:22:20.712104 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.712108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.714662 (+ 2554us) server_negotiation.cc:407] Connection header received
0808 18:22:20.714829 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.714833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.714881 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.714932 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.715452 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52892: BlockingRecv error: recv got EOF from 127.0.0.1:52892 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 18:22:20.716933 31208 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.712605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.27.18.252:41085)
0808 18:22:20.712736 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.712740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.716048 (+ 3308us) server_negotiation.cc:407] Connection header received
0808 18:22:20.716243 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.716246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.716291 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.716354 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.716818 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51944: BlockingRecv error: recv got EOF from 127.0.0.1:51944 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 18:22:20.721203 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.719640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.27.18.253:32791)
0808 18:22:20.719730 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.719734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.720277 (+ 543us) server_negotiation.cc:407] Connection header received
0808 18:22:20.720506 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.720510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.720582 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.720638 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.721122 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52904: BlockingRecv error: recv got EOF from 127.0.0.1:52904 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 18:22:20.726065 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.724664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52906 (local address 127.27.18.253:32791)
0808 18:22:20.724772 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.724778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.725225 (+ 447us) server_negotiation.cc:407] Connection header received
0808 18:22:20.725424 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.725427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.725473 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.725522 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.725984 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52906: BlockingRecv error: recv got EOF from 127.0.0.1:52906 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 18:22:20.731529 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.730022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.27.18.253:32791)
0808 18:22:20.730117 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.730120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.730607 (+ 487us) server_negotiation.cc:407] Connection header received
0808 18:22:20.730777 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.730780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.730822 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.730885 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.731352 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52908: BlockingRecv error: recv got EOF from 127.0.0.1:52908 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 18:22:20.738857 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.737500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.27.18.253:32791)
0808 18:22:20.737615 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.737621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.738060 (+ 439us) server_negotiation.cc:407] Connection header received
0808 18:22:20.738251 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.738254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.738295 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.738352 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.738782 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52924: BlockingRecv error: recv got EOF from 127.0.0.1:52924 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 18:22:20.749790 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.748774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.27.18.253:32791)
0808 18:22:20.748853 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.748856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.749081 (+ 225us) server_negotiation.cc:407] Connection header received
0808 18:22:20.749152 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.749155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.749196 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.749273 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.749683 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52934: BlockingRecv error: recv got EOF from 127.0.0.1:52934 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 18:22:20.768250 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.767256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52944 (local address 127.27.18.253:32791)
0808 18:22:20.767398 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.767403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.767539 (+ 136us) server_negotiation.cc:407] Connection header received
0808 18:22:20.767613 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.767616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.767675 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.767760 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.768149 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52944: BlockingRecv error: recv got EOF from 127.0.0.1:52944 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 18:22:20.802827 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.801753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.27.18.253:32791)
0808 18:22:20.801930 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.801933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.802004 (+ 71us) server_negotiation.cc:407] Connection header received
0808 18:22:20.802108 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.802110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.802156 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.802223 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.802706 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52946: BlockingRecv error: recv got EOF from 127.0.0.1:52946 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 18:22:20.870031 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.868802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.27.18.253:32791)
0808 18:22:20.868959 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.868964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:20.869104 (+ 140us) server_negotiation.cc:407] Connection header received
0808 18:22:20.869231 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:20.869234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:20.869291 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:20.869368 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:20.869885 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52956: BlockingRecv error: recv got EOF from 127.0.0.1:52956 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 18:22:21.000941 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:20.999827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.27.18.253:32791)
0808 18:22:20.999959 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:20.999963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.000183 (+ 220us) server_negotiation.cc:407] Connection header received
0808 18:22:21.000293 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.000302 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.000367 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.000433 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:21.000821 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52964: BlockingRecv error: recv got EOF from 127.0.0.1:52964 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 18:22:21.259997 31094 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 18:22:21.258929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52974 (local address 127.27.18.253:32791)
0808 18:22:21.259099 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:21.259103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.259118 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:21.259251 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.259254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.259311 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.259451 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 18:22:21.259858 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52974: BlockingRecv error: recv got EOF from 127.0.0.1:52974 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 18:22:21.555109 31231 catalog_manager.cc:1259] Loaded cluster ID: 557676453e254f36975cc02df653836a
I20260808 18:22:21.555177 31231 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: loading cluster ID for follower catalog manager: success
I20260808 18:22:21.556746 31231 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: acquiring CA information for follower catalog manager: success
I20260808 18:22:21.557897 31231 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8517eae9daf6479db25be05e4a7283ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:22:21.574749 31259 catalog_manager.cc:1259] Loaded cluster ID: 557676453e254f36975cc02df653836a
I20260808 18:22:21.574815 31259 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: loading cluster ID for follower catalog manager: success
I20260808 18:22:21.576226 31259 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: acquiring CA information for follower catalog manager: success
I20260808 18:22:21.577301 31259 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3dbb9acb276e4a68897a90705de6c21b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 18:22:21.777464 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:21.773797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52976 (local address 127.27.18.253:32791)
0808 18:22:21.774028 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:21.774031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.774046 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:21.774174 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.774176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.774295 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.774382 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:21.774587 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.775041 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.775760 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.776145 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.776250 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:21.776540 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:21.776994 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:21.777178 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:21.777215 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 18:22:21.777270 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":67,"threads_started":1}
I20260808 18:22:21.783445 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:21.780171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.27.18.252:41085)
0808 18:22:21.780407 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:21.780412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.780428 (+ 16us) server_negotiation.cc:407] Connection header received
0808 18:22:21.780574 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.780577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.780673 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.780781 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:21.780999 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.781432 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.782143 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.782342 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.782438 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:21.782697 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:21.783018 (+ 321us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:21.783148 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:21.783204 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 18:22:21.783253 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":60,"threads_started":1}
I20260808 18:22:21.789919 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:21.786078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.27.18.253:32791)
0808 18:22:21.786223 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:21.786228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.786651 (+ 423us) server_negotiation.cc:407] Connection header received
0808 18:22:21.786754 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.786757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.786797 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.786877 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:21.787122 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.787876 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.788877 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.789156 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.789234 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:21.789512 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:21.789576 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:21.789639 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:21.789749 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 18:22:21.789794 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 18:22:21.790118 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:21.786485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.27.18.252:41085)
0808 18:22:21.786580 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:21.786584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.787011 (+ 427us) server_negotiation.cc:407] Connection header received
0808 18:22:21.787124 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.787126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.787172 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.787239 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:21.787526 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.788185 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.789098 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.789348 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.789396 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:21.789715 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:21.789778 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:21.789834 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:21.789963 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 18:22:21.790010 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 18:22:21.790169 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 18:22:21.786041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.27.18.254:46419)
0808 18:22:21.786369 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 18:22:21.786374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 18:22:21.786389 (+ 15us) server_negotiation.cc:407] Connection header received
0808 18:22:21.786470 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 18:22:21.786473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 18:22:21.786604 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 18:22:21.786736 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 18:22:21.786963 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.787723 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.788626 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 18:22:21.788942 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 18:22:21.789038 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 18:22:21.789342 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 18:22:21.789582 (+ 240us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 18:22:21.789670 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 18:22:21.789983 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 18:22:21.790044 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":59,"threads_started":1}
I20260808 18:22:21.792964 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30859
I20260808 18:22:21.809974 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 30975
I20260808 18:22:21.819515 27723 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPL43Mv/build/debug/bin/kudu with pid 31095
2026-08-08T18:22:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3144 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6304 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283381 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST